lighty's life

I’m sure you know what “Crawl-Delay” is, but you may or may not know that, not all search engine crawlers support this nice stuff.

What to do for those don’t obey the instrustion? They’ll eat all your Mbits/month or slow your webserver down. OK, ban it with url.access-deny. This is the only option u can choose before. But you don’t want to remove your pages from the stupid search engine index, do you?

Here comes another option for you: with this patch, u can delay handling of a specified request for some seconds. Example configuration:

$HTTP["user-agent"] =~ "stupid-crawler" {
    connection.delay-seconds = 2
}

OK, here’s the link to the lighttpd-2296-request-handle-delay.patch which applies to branches/lighttpd-1.4.x@2296

Be aware that this patch is to be reviewed before commited to repo.

It’s been some time since the last 1.4 release and especially since the last 1.5 prerelease. There have been a lot of fixes in both branches so be sure to check them out, no matter if you are using 1.4 or 1.5.

You can get the prereleases from this url: http://opensu.se/~darix/lighttpd/

The dev team has gotten some new members recently that you might not know of. Here’s a heads up (in alphabetical order):

• hoffie
• icy
• nitrox
• stbuehler

If you hang out in #lighttpd, you will already know these names.

That being said, there is also some news about the state of the development of lighty.

Lighty is a great webserver and we all love it. But if you use it for quite some time, you’ll eventually find out that it’s not perfect. There are some oddities and shortcomings that cannot be ironed out without rewriting a significant part of the core.

This is where I had the idea to start all over. With lessons learned from the past, we could write a new version of lighty that fixes the design issues the current versions have. But at the same time keep all the good stuff and remain fairly compatible. The main idea to be fast and lightweight is still valid and if you are familiar with 1.4.x or 1.5, you will have no problems with the new version.

Jan had the idea to use glib for stuff like strings/buffers and arrays. He wanted to rewrite the corresponding parts and did so in a branch of his. I proposed to take this even further and use glib throughout the source. It makes coding easier, faster and in the end more secure because you can rely on proven and well tested source.

stbuehler jumped in and created a new branch where he started to hack on an all new lighty using the ideas from above. The following days we discussed a lot – and still do so – about the new design. What we could do better, what shortcomings we wanted to fix.

The result is nothing less but a more flexible and faster design for lighty.

This sounds awesome but where is the catch you might think. Well, currently we have not too much code ready but we are actively hacking on it. Do not ask when it will be released, there is no estimated date. We are still at the very beginning of the new version but hope to have it some day become the official Lighttpd 2.0

We created a page that lists some of the (technical) plans we currently have. You might find some of them interesting.

This is it for today folks. Hope you like the route we are taking and maybe drop us a line in #lighttpd (freenode) or in the comments.

- update -

To make it clear: 1.5 or 1..4 have not been dropped. If you think 1.5 is unstable and you get a segfault or something like that, then tell us. File a ticket, attach a stacktrace and document the problem in a detailed way so it can be fixed. Thanks for using lighty.

After quite some time lighttpd is moving forward again. Please give this pre-release a try and check if it drops in nicely and doesn’t cause any new issues.

It is mostly a bug-release and we want to make sure it is a good release. If this pre-release passes your tests, we’ll put the real release online in a few days.

Download: http://opensu.se/~darix/lighttpd/

ChangeLog: http://trac.lighttpd.net/trac/browser/branches/lighttpd-1.4.x/NEWS

Google is doing a the Summer of Code again and I was approached from different sides to participate this year. There is already a lot going on in SVN again working towards a new 1.4.19 release, but for the future we want to attract more developers for more new, great features.

Before I submit the application request I would like to collect some more ideas what we would like to see to be done. At GSoC2008/Ideas I added my short list of projects that I would like to mentor.

If you would like to be a mentor too, please add your own list.

• Documentation improvements
• Integration into web-panels
• maintaining the OpenWRT port

are just some small ideas where we could take help from non-developers.

Part of my ongoing, hidden effort to port lighttpd to glib I can now at least say: it compiles.

Well, mostly. The core does compile and mod_staticfile is also mostly done. Mostly, as I don’t have decided on how to handle the config-file handling yet.

With the first implementation of mod_cml and later mod_magnet smart users asked why we don’t use lua also as config language for lighttpd.

Up to now I was pushing back and saying that a huge change in the configfile would make upgraders very unhappy as they would have to rewrite the config files. Looking at what we have for 1.5.0 right now, we need tweaks anyway.

As part of a weekend project I’m ripping 1.5 apart a bit. I worked with glib for quite some time now and feel the need to replace the base-structures around “buffer” (strings) and “array” (hash) apart and replace it with the glib counter-parts.

To make this a nice experience and that I don’t harm anyone else I imported trunk into a local bzr tree:

bzr clone svn://svn.lighttpd.net/lighttpd/

to clone the SVN tree.

Let’s see how well that experiment goes.

darix released 1.4.18 which contains a fix for a buffer-overrun in the fastcgi protocol.

Some time passed since the last pre-release, time for an update.

Everyone who runs 1.5.0 already has to upgrade to get fixes several vulnerabilities that got fixed in the 1.4.x branch already.

If this release passes your requirements it will be the last 1.5.0 pre-release. Afterwards we will start the the normal 1.5.x series and will add the missing features in 1.5.1 and later.

Download: http://www.lighttpd.net/download/lighttpd-1.5.0-r1992.tar.gz

md5sum: b62e2442ee0f3395844b54385b14397a

Changes

• added native support for mingw32
• added experimental option to compile without glib
• fixed mod_uploadprogress
• fixed endless loop on freebsd-sendfile (#1289)
• fixed compile in IRIX and HP/UX
• fixed hardcoded font-sizes in mod_dirlisting (#1267)
• fixed different ETag length on 32/64 platforms (#1279)
• fixed conditional dir-listing.exclude (#930)
• fixed CONTENT_LENGTH = -1 in mod_cgi (#1276)
• fixed typecast of NULL on execl() (#1235)
• fixed extra Content-Length header on 1xx, 204 and 304 (#1002)
• fixed mysql server reconnects (#518)
• fixed prctl() usage (#1310, #1333)
• fixed FastCGI header overrun in mod_proxy_backend_fastcgi (reported by mattias@secweb.se)
• fixed mem-leak in mod_auth (reported by Stefan Esser)
• fixed crash with md5-sess and cnonce not set in mod_auth (reported by Stefan Esser)
• fixed missing check for base64 encoded string in mod_auth and Basic auth (reported by Stefan Esser)
• fixed possible crash in Auth-Digest header parser on trailing WS in mod_auth (reported by Stefan Esser)