lighty's life: Category lighttpd

Growing the team

icy — Wed, 23 Jul 2008 14:43:00 +0000

The dev team has gotten some new members recently that you might not know of. Here’s a heads up (in alphabetical order):

hoffie
icy
nitrox
stbuehler

If you hang out in #lighttpd, you will already know these names.

That being said, there is also some news about the state of the development of lighty.

Lighty is a great webserver and we all love it. But if you use it for quite some time, you’ll eventually find out that it’s not perfect. There are some oddities and shortcomings that cannot be ironed out without rewriting a significant part of the core.

This is where I had the idea to start all over. With lessons learned from the past, we could write a new version of lighty that fixes the design issues the current versions have. But at the same time keep all the good stuff and remain fairly compatible. The main idea to be fast and lightweight is still valid and if you are familiar with 1.4.x or 1.5, you will have no problems with the new version.

Jan had the idea to use glib for stuff like strings/buffers and arrays. He wanted to rewrite the corresponding parts and did so in a branch of his. I proposed to take this even further and use glib throughout the source. It makes coding easier, faster and in the end more secure because you can rely on proven and well tested source.

stbuehler jumped in and created a new branch where he started to hack on an all new lighty using the ideas from above. The following days we discussed a lot – and still do so – about the new design. What we could do better, what shortcomings we wanted to fix.

The result is nothing less but a more flexible and faster design for lighty.

This sounds awesome but where is the catch you might think. Well, currently we have not too much code ready but we are actively hacking on it. Do not ask when it will be released, there is no estimated date. We are still at the very beginning of the new version but hope to have it some day become the official Lighttpd 2.0

We created a page that lists some of the (technical) plans we currently have. You might find some of them interesting.

This is it for today folks. Hope you like the route we are taking and maybe drop us a line in #lighttpd (freenode) or in the comments.

PRE-RELEASE: lighttpd-1.4.19-r2118

jan — Wed, 05 Mar 2008 21:48:00 +0000

After quite some time lighttpd is moving forward again. Please give this pre-release a try and check if it drops in nicely and doesn’t cause any new issues.

It is mostly a bug-release and we want to make sure it is a good release. If this pre-release passes your tests, we’ll put the real release online in a few days.

Download: http://opensu.se/~darix/lighttpd/

ChangeLog: http://trac.lighttpd.net/trac/browser/branches/lighttpd-1.4.x/NEWS

GSoC: collecting ideas

jan — Tue, 04 Mar 2008 11:25:00 +0000

Google is doing a the Summer of Code again and I was approached from different sides to participate this year. There is already a lot going on in SVN again working towards a new 1.4.19 release, but for the future we want to attract more developers for more new, great features.

Before I submit the application request I would like to collect some more ideas what we would like to see to be done. At GSoC2008/Ideas I added my short list of projects that I would like to mentor.

If you would like to be a mentor too, please add your own list.

Documentation improvements
Integration into web-panels
maintaining the OpenWRT port

are just some small ideas where we could take help from non-developers.

Project Glib: it compiles ... mostly

jan — Tue, 19 Feb 2008 17:14:00 +0000

Part of my ongoing, hidden effort to port lighttpd to glib I can now at least say: it compiles.

Well, mostly. The core does compile and mod_staticfile is also mostly done. Mostly, as I don’t have decided on how to handle the config-file handling yet.

So, what has been done so far:

buffer.c and buffer.h are gone and are replaced by GString
array.c is gone and is replace by GHashTable or GPtrArray
array.h will vanish when the data_…c goes away
log.c is now a wrapper around g_log_…

As you see this is ripping out the basic data-structures and replaces them by a glib look-alike.

This isn’t just a drop-in replacement as the structures use other field-names and sometimes have a slightly different meaning.

One example is GString vs. buffer:

buffer {
  char *ptr;
  size_t used;
  size_t size;
};

GString {
  char *str;
  size_t len;
  size_t allocated_len;
};

While str and ptr, and allocated_len and size match, used and len differ by one byte (used includes the NUL-byte, len doesn’t).

One side-effect of this boring work is seeing the code-size shrinking, piece by piece.

Seeing old, sometimes dirty code vanish is another nice side-effect. Like the log_error_write() call … man …

Weekends Projects: lua as config-language

jan — Sat, 09 Feb 2008 23:09:00 +0000

With the first implementation of mod_cml and later mod_magnet smart users asked why we don’t use lua also as config language for lighttpd.

Up to now I was pushing back and saying that a huge change in the configfile would make upgraders very unhappy as they would have to rewrite the config files. Looking at what we have for 1.5.0 right now, we need tweaks anyway.

This always happens if you rip a thing apart: you see more things that should be cleaned up. My first experiments are a bit done:

server = {
        name = "my.example.org",
        modules = {
                "mod_indexfile" 
        },
        ["errorfile-prefix"] = "foobar",
}

sockets = {
        [":80"] = {
                server = { name = "foo.example.org" },
                HTTP = { url = function(url)
                        if url == "foo" then
                                server.name = "baz.example.org" 
                        end
                end },
        }
}

sockets[":80"].HTTP.url("foo")
print(server.name)

This is a first idea. It shows how the LUA syntax isn’t sooo different from our current config syntax.

The global variables, stay global as now.
socket conditionals is now a simple array of sockets
conditionals are functions, now you can do what you want with it

I hope that I can handle the full config syntax into this structure. For now it is looks more appealing that ripping the old config-parser code apart and turn it into glib-notation. With all the conditionals and types we basicly have implemented our own scripting language.

Let’s see where this leads too.

Weekend Projects: killing buffer.c and array.c

jan — Sat, 09 Feb 2008 08:46:00 +0000

As part of a weekend project I’m ripping 1.5 apart a bit. I worked with glib for quite some time now and feel the need to replace the base-structures around “buffer” (strings) and “array” (hash) apart and replace it with the glib counter-parts.

To make this a nice experience and that I don’t harm anyone else I imported trunk into a local bzr tree:

bzr clone svn://svn.lighttpd.net/lighttpd/

to clone the SVN tree.

Let’s see how well that experiment goes.

ANNOUNCE: lighttpd 1.4.18

jan — Sun, 09 Sep 2007 21:43:00 +0000

darix released 1.4.18 which contains a fix for a buffer-overrun in the fastcgi protocol.

PRE-RELEASE: lighttpd-1.5.0-r1992

jan — Thu, 06 Sep 2007 22:21:00 +0000

Some time passed since the last pre-release, time for an update.

Everyone who runs 1.5.0 already has to upgrade to get fixes several vulnerabilities that got fixed in the 1.4.x branch already.

If this release passes your requirements it will be the last 1.5.0 pre-release. Afterwards we will start the the normal 1.5.x series and will add the missing features in 1.5.1 and later.

Download: http://www.lighttpd.net/download/lighttpd-1.5.0-r1992.tar.gz

md5sum: b62e2442ee0f3395844b54385b14397a

Changes

added native support for mingw32
added experimental option to compile without glib
fixed mod_uploadprogress
fixed endless loop on freebsd-sendfile (#1289)
fixed compile in IRIX and HP/UX
fixed hardcoded font-sizes in mod_dirlisting (#1267)
fixed different ETag length on 32/64 platforms (#1279)
fixed conditional dir-listing.exclude (#930)
fixed CONTENT_LENGTH = -1 in mod_cgi (#1276)
fixed typecast of NULL on execl() (#1235)
fixed extra Content-Length header on 1xx, 204 and 304 (#1002)
fixed mysql server reconnects (#518)
fixed prctl() usage (#1310, #1333)
fixed FastCGI header overrun in mod_proxy_backend_fastcgi (reported by mattias@secweb.se)
fixed mem-leak in mod_auth (reported by Stefan Esser)
fixed crash with md5-sess and cnonce not set in mod_auth (reported by Stefan Esser)
fixed missing check for base64 encoded string in mod_auth and Basic auth (reported by Stefan Esser)
fixed possible crash in Auth-Digest header parser on trailing WS in mod_auth (reported by Stefan Esser)

There is an Angel for lighty

jan — Sun, 02 Sep 2007 22:36:00 +0000

In changeset [1981] I added a angel process to the lighttpd build. It solves several problems when we have everything running as expected:

SIGHUP leads to a graceful restart (config reloads)
SIGINT is graceful shutdown as now
all unhandled signals lead to a restart of the lighttpd process

We have to be careful with the angel as it will stay alive and stay running as the user it is started as (usually root). That’s the only way we can restart lighttpd from scratch on restart.

This will also solve other problems in 1.5.0:

bug #1271, unsafe logfiles
... and perhaps others

The angel has to implement some security measures to stay clean:

the angel can only start the lighttpd binary which is compiled into the angel at compile time
if run as root the starting user has to be root too or be in the same group as the owner of the lighttpd binary
... and perhaps more restrictions

As extra features we will add ulimit support to let lighttpd die in case it is using to much memory. But that’s for later.

lighttpd at the Kieler LinuxTage

jan — Tue, 28 Aug 2007 11:43:55 +0000

I’ll give a presentation on lighttpd at the Kieler Linuxtage on September 8th, 2007. I’ll talk about lighttpd, its past and future, the special modules and performance.

I will also try to give some content to the un-conference going on at the same time. Perhaps some MySQL Proxy stuff ?