The New mod_proxy_core 9

Posted by jan Sat, 15 Jul 2006 21:06:00 GMT

One thing I heard from everyone at the railsconf was: Please fix mod_proxy. In my talk I presented a look into the future and how mod_proxy will evolve in lighty 1.5.x, unification of the backend-plugin, .,.

Well, perhaps it will be a bit earlier than 1.5.0

Over the last week and especially at this weekend I make big steps into the right direction:

  • a string HTTP-response parser based on lemon used by all backend plugins (FastCGI, CGI, SCGI, Proxy)
  • all backends use a sliding window to send data. The data-stream in chunked and as soon as a chunk is finished it is freed or reused.
  • the network-code is only implemented once

But more important is the work on mod_proxy_core with this feature-set:

  • Unified core for all backends * failover, balancers, config, streaming, ... are shared
  • several protocols for FastCGI, CGI, SCGI and HTTP
  • Keep-Alive for FastCGI and HTTP
  • HTTP/1.1 (incl. chunked encoding to the backend)
  • queueing of connections in the plugin (instead of using the kernel for it)
  • HTTP over SSL to the backends
  • Host-Selection by hostname (auto-loadbalancing for multi-homed hosts)
  • Request/Response rewrite in the proxy-module before they are sent to the backends

What is working up to now ?

Up to now I can use RR-balancing via HTTP/1.1 with keep-alive and chunked-encoding. As testbase I used http://cnn.com/ and http://www.stern.de/ for the RR-testing and http://php.net/ to test chunked encoding. All of them are contacted via HTTP/1.1 and use Keep-Alive.

I only specified the hostname, lighty resolved the hostname at startup and added all IPs to the LB-pool. .oO( Hey, it would be easy to add and remove IPs from this pool at runtime )

It was good to kill some old code on the way. The fd-event handling got a cleanup and more flexible in the handling. the old backend-plugins got simplified as the share more code, ... over all good weekend so far.

What is missing ?

Failover handling and queueing is planned for tomorrow, process-spawning will be unified a bit later. mod_proxy_core will be part of 1.4.12 next to the old, well-known plugins for FastCGI, SCGI, CGI and HTTP.

When the code for mod_proxy_core is commited I advise everyone to take a look at the code for it. It is very clean and well-documented and should be easy to read. ... if you know C. :)

Trackbacks

Use the following link to trackback from your own site:
http://blog.lighttpd.net/articles/trackback/1758

  1. 500 errors when using Mongrel and Pound From I am rice
    We had some problems with 500 errors occurring in a project when running on Mongrel and Pound. Turns out (as David has also mentioned) it's dude to Pound's default timeout being set to 15 seconds. So if you've got any intensive apps, increase the ...
  2. PRE-RELEASE: lighttpd-1.4.12-20060724-0947.tar.gz From lighty's life
    I just uploaded the 3rd pre-release of lighttpd 1.4.12: http://www.lighttpd.net/download/lighttpd-1.4.12-20060724-0947.tar.gz This pre-release should work on most platforms and it mainly got improvements for our mongrel users. A small test has sho...
  3. Rails, Mongrel, Lighty and Mint From Blog of Paul Ingles
    Well, after a few problems with getting Gems working yesterday I appear to have made some progress. As Dave mentioned, it indeed turned out to be a memory problem. Stopping all my sites and MySQL running seemed to do the trick, so I managed to suc...
Comments

Leave a response

  1. Sebastian Sat, 15 Jul 2006 21:30:56 GMT
    This is great news. Lighty + mod_proxy + Mongrel will be a splendid solution for Rails deployment and beyond; all that reactionary Apache talk was sure making me feel queasy. :) btw, Jan, it was very nice meeting you at RailsConf. Thanks for all the great work.
  2. Damien Tanner Sat, 15 Jul 2006 23:00:24 GMT
    Great news indeed, I among many will look forward to the new mod_proxy!
  3. chernousov Sun, 16 Jul 2006 07:22:53 GMT
    Jan, those changes are great, but where is ticket #755 "fastcgi can't handle response 8k"? I made it 14/jul, but now it's deleted - check http://trac.lighttpd.net/trac/ticket/755 This error is very critical, it blocks any work with fastcgi backend!
  4. Uwe Sun, 16 Jul 2006 07:48:33 GMT
    That sounds really great. If you are working on mod_proxy, can you please add compression support to it? Thank you very much
  5. Jan Kneschke Sun, 16 Jul 2006 08:18:18 GMT
    chernousov: can you open the ticket again please. I removed the wrong ticket-id when I was cleaning up the ticket-spam. Sorry about that.
  6. chernousov Sun, 16 Jul 2006 09:02:51 GMT
    Jan, check it please: http://trac.lighttpd.net/trac/ticket/754 Sorry for this discussion HERE but i don't know better way to get your attention to this bug. (Btw, seems my account from trac is deleted and i'm not able to register again. http://trac.lighttpd.net/trac/register gives me 404)
  7. anonymous coward Sun, 16 Jul 2006 10:03:52 GMT
    Is there something like ProxyPreserveHost available in lighty?
  8. Tyler Sun, 16 Jul 2006 23:39:42 GMT
    Count me among those eager to start testing this as soon as 1.4.12 is released.
  9. Jan Kneschke Sat, 22 Jul 2006 09:32:34 GMT
    ProxyPreserveHost is our default behaviour. You have to rewrite the Host header explicitly.
Comments