lighty's life: The New mod_proxy_core

The New mod_proxy_core

jan — Sat, 15 Jul 2006 21:06:00 +0000

One thing I heard from everyone at the railsconf was: Please fix mod_proxy. In my talk I presented a look into the future and how mod_proxy will evolve in lighty 1.5.x, unification of the backend-plugin, .,.

Well, perhaps it will be a bit earlier than 1.5.0

Over the last week and especially at this weekend I make big steps into the right direction:

a string HTTP-response parser based on lemon used by all backend plugins (FastCGI, CGI, SCGI, Proxy)
all backends use a sliding window to send data. The data-stream in chunked and as soon as a chunk is finished it is freed or reused.
the network-code is only implemented once

But more important is the work on mod_proxy_core with this feature-set:

Unified core for all backends * failover, balancers, config, streaming, ... are shared
several protocols for FastCGI, CGI, SCGI and HTTP
Keep-Alive for FastCGI and HTTP
HTTP/1.1 (incl. chunked encoding to the backend)
queueing of connections in the plugin (instead of using the kernel for it)
HTTP over SSL to the backends
Host-Selection by hostname (auto-loadbalancing for multi-homed hosts)
Request/Response rewrite in the proxy-module before they are sent to the backends

What is working up to now ?

Up to now I can use RR-balancing via HTTP/1.1 with keep-alive and chunked-encoding. As testbase I used http://cnn.com/ and http://www.stern.de/ for the RR-testing and http://php.net/ to test chunked encoding. All of them are contacted via HTTP/1.1 and use Keep-Alive.

I only specified the hostname, lighty resolved the hostname at startup and added all IPs to the LB-pool. .oO( Hey, it would be easy to add and remove IPs from this pool at runtime )

It was good to kill some old code on the way. The fd-event handling got a cleanup and more flexible in the handling. the old backend-plugins got simplified as the share more code, ... over all good weekend so far.

What is missing ?

Failover handling and queueing is planned for tomorrow, process-spawning will be unified a bit later. mod_proxy_core will be part of 1.4.12 next to the old, well-known plugins for FastCGI, SCGI, CGI and HTTP.

When the code for mod_proxy_core is commited I advise everyone to take a look at the code for it. It is very clean and well-documented and should be easy to read. ... if you know C. :)

"The New mod_proxy_core" by Jan Kneschke

Sat, 22 Jul 2006 09:32:34 +0000

ProxyPreserveHost is our default behaviour. You have to rewrite the Host header explicitly.

"The New mod_proxy_core" by Tyler

Sun, 16 Jul 2006 23:39:42 +0000

Count me among those eager to start testing this as soon as 1.4.12 is released.

"The New mod_proxy_core" by anonymous coward

Sun, 16 Jul 2006 10:03:52 +0000

Is there something like ProxyPreserveHost available in lighty?

"The New mod_proxy_core" by chernousov

Sun, 16 Jul 2006 09:02:51 +0000

Jan, check it please: http://trac.lighttpd.net/trac/ticket/754 Sorry for this discussion HERE but i don't know better way to get your attention to this bug. (Btw, seems my account from trac is deleted and i'm not able to register again. http://trac.lighttpd.net/trac/register gives me 404)

"The New mod_proxy_core" by Jan Kneschke

Sun, 16 Jul 2006 08:18:18 +0000

chernousov: can you open the ticket again please. I removed the wrong ticket-id when I was cleaning up the ticket-spam. Sorry about that.

"The New mod_proxy_core" by Uwe

Sun, 16 Jul 2006 07:48:33 +0000

That sounds really great. If you are working on mod_proxy, can you please add compression support to it? Thank you very much

"The New mod_proxy_core" by chernousov

Sun, 16 Jul 2006 07:22:53 +0000

Jan, those changes are great, but where is ticket #755 "fastcgi can't handle response 8k"? I made it 14/jul, but now it's deleted - check http://trac.lighttpd.net/trac/ticket/755 This error is very critical, it blocks any work with fastcgi backend!

"The New mod_proxy_core" by Damien Tanner

Sat, 15 Jul 2006 23:00:24 +0000

Great news indeed, I among many will look forward to the new mod_proxy!

"The New mod_proxy_core" by Sebastian

Sat, 15 Jul 2006 21:30:56 +0000

This is great news. Lighty + mod_proxy + Mongrel will be a splendid solution for Rails deployment and beyond; all that reactionary Apache talk was sure making me feel queasy. :) btw, Jan, it was very nice meeting you at RailsConf. Thanks for all the great work.