lighty's life: Project Glib: it compiles ... mostly

Project Glib: it compiles ... mostly

jan — Tue, 19 Feb 2008 17:14:00 +0000

Part of my ongoing, hidden effort to port lighttpd to glib I can now at least say: it compiles.

Well, mostly. The core does compile and mod_staticfile is also mostly done. Mostly, as I don’t have decided on how to handle the config-file handling yet.

So, what has been done so far:

buffer.c and buffer.h are gone and are replaced by GString
array.c is gone and is replace by GHashTable or GPtrArray
array.h will vanish when the data_…c goes away
log.c is now a wrapper around g_log_…

As you see this is ripping out the basic data-structures and replaces them by a glib look-alike.

This isn’t just a drop-in replacement as the structures use other field-names and sometimes have a slightly different meaning.

One example is GString vs. buffer:

buffer {
  char *ptr;
  size_t used;
  size_t size;
};

GString {
  char *str;
  size_t len;
  size_t allocated_len;
};

While str and ptr, and allocated_len and size match, used and len differ by one byte (used includes the NUL-byte, len doesn’t).

One side-effect of this boring work is seeing the code-size shrinking, piece by piece.

Seeing old, sometimes dirty code vanish is another nice side-effect. Like the log_error_write() call … man …

"Project Glib: it compiles ... mostly" by Henrik Holst

Thu, 13 Mar 2008 16:51:36 +0000

That is true, but only if your one malloc() was for an insane amount of pages. Of course one can turn of the overcommit quite easily, but I think that you would be caught by bugs in quite a few applications then. I think that the reason behind glib2's use of abort() is that if you receive NULL from malloc() you have a 99% chance that the heap is corrupted and you better abort your process. The situation that Tom described where lighty would be terminated when handling to many simultaneous connections is one situation where NULL would practically never ever be returned by malloc() due to the overcommit in linux.

"Project Glib: it compiles ... mostly" by Timo Savola

Thu, 13 Mar 2008 15:37:59 +0000

Henrik: Actually memory allocation on Linux can fail if you try to allocate too much over the top (see linux/Documentation/vm/overcommit-accounting).

"Project Glib: it compiles ... mostly" by Henrik Holst

Thu, 13 Mar 2008 09:52:58 +0000

Tom: Memory allocation in Linux does not return NULL on out-of-memory, libc will only return NULL if it detected a corrupted heap. Linux will only perform the allocation itself when you actually use the memory so there is no way to tell you on malloc() time that you are out of memory, so calling abort() on failed allocation is the only sane way to do it so glib2 performs correct.

"Project Glib: it compiles ... mostly" by Jason

Wed, 12 Mar 2008 15:49:05 +0000

jan, Can you provide some insight on why the move the glib?v Thanks.

"Project Glib: it compiles ... mostly" by Marco

Tue, 11 Mar 2008 23:49:15 +0000

Yeah, that's great work. Keep on. Never had any problems with lightty and it's serving plenty of gigabytes a day for months.

"Project Glib: it compiles ... mostly" by Reed A. Cartwright

Fri, 29 Feb 2008 07:09:27 +0000

re: tom@dbservice.com Well there is a g_try_malloc, that will return NULL if the memory allocation fails and does not abort.

"Project Glib: it compiles ... mostly" by Anders

Thu, 28 Feb 2008 15:10:59 +0000

That's great, I've been using GLib myself in my own app and it makes C a lot nicer to work with!

"Project Glib: it compiles ... mostly" by lu_zero

Thu, 28 Feb 2008 08:32:42 +0000

Having too many clients to be serviced shouldn't cause the whole server to close, but just to reject newer connections...

"Project Glib: it compiles ... mostly" by jasper

Wed, 27 Feb 2008 23:56:36 +0000

tom: Why o' why do you have a "24/7" web server that has so little memory that memory allocation fails? You would have to both be completely out of physical RAM, and also have no swap left.

"Project Glib: it compiles ... mostly" by tom@dbservice.com

Tue, 26 Feb 2008 10:44:57 +0000

Why o' why glib? Don't you know that glib abort()s when memory allocation fails? That's really bad for a webserver that's supposed to run 24/7

"Project Glib: it compiles ... mostly" by lu_zero

Sat, 23 Feb 2008 12:07:02 +0000

I'm currently using part of your design and currently part of your nice syntax parser in feng (a small rtsp streaming server), since it is already using glib I was about to start converting some stuff the way you already did ^^. Thank you

"Project Glib: it compiles ... mostly" by Jan (the other one)

Wed, 20 Feb 2008 17:56:43 +0000

Yeah and camelcase instead looks nicer than lower case :) In a few years we have IDEs, which provice AI refactoring. Just describe your wishes and the IDE does the boring work for you :) Have you done some performance benchmarks yet? I'm excited to see the results :) Great work!