lighty's life

The implementation for the feature is a bit different than it was done in mod_fastcgi.

The old implementation only replaced the content-body with the static-file and sent it out. The new implementation does it slightly different and gains a whole set of benifits.

When the X-Sendfile header is detected (and is allowed in the config) the content-body is ignored and a internal redirect is done. mod_proxy_core takes itself out of the loop and mod_staticfile takes over the request.

mod_staticfile can do all the magic:

• setting Last-Modified and ETag
• handling ‘304 Not Modified’
• handling Range requests
• compression

Setup

The setup is as before:

$HTTP["url"] =~ "^/bugme(/|$)" {
    proxy-core.balancer = "round-robin" 
    proxy-core.protocol = "http" 
    proxy-core.backends = ( "127.0.0.1:2000" )
    proxy-core.allow-x-sendfile = "enable" 
}

On port 2000 I have a small $ nc -l 2000 running to simulate a super complex, high secure application. $ wget --header='Accept-Encoding: gzip' http://127.0.0.1:1025/bugme/upload.html is my browser replacement:

GET /bugme/upload.html HTTP/1.0
X-Forwarded-For: 127.0.0.1
X-Host: 127.0.0.1:1025
X-Forwarded-Proto: http
User-Agent: Wget/1.10.2 (Red Hat modified)
Accept: */*
Host: 127.0.0.1:1025
Accept-Encoding: gzip
FOO: foo

HTTP/1.0 200 OK
X-Sendfile: /path/to/upload.html

... and there we have the compressed upload.html:

$ ls -l upload.html
-rw-rw-r--  1 jan jan 661 Sep 26  2005 upload.html
$ ls -l /path/to/upload.html
-rw-rw-r--  1 jan jan 1193 Sep 26  2005 /path/to/upload.html

All you have to do in your application is setting the X-Sendfile header:

<?php
  header("X-Sendfile: /path/to/upload.html");
?>

or in Rails:

response.headers["X-Sendfile"] = "/path/to/upload.html" 

I’ve taken wikipedia as testbed for the hash-balancing:

$SERVER["socket"] == ":1445" {
  proxy-core.balancer = "hash" 
  proxy-core.protocol = "http" 
  proxy-core.backends = ( "wikipedia.org" )
  proxy-core.rewrite-response = (
    "Location" => ( "^http://en.wikipedia.org/(.*)" => "http://127.0.0.1:1445/$1" ),
  )
  proxy-core.rewrite-request = (
    "Host" => ( ".*" => "en.wikipedia.org" ),
 )
}

The domain wikipedia.org resolves to several IP-addresses:

(trace) resolving wikipedia.org on port 80
(trace) adding 207.142.131.204:80 to the address-pool
(trace) adding 207.142.131.205:80 to the address-pool
(trace) adding 207.142.131.206:80 to the address-pool
(trace) adding 207.142.131.210:80 to the address-pool
(trace) adding 207.142.131.213:80 to the address-pool
(trace) adding 207.142.131.214:80 to the address-pool
(trace) adding 207.142.131.235:80 to the address-pool
(trace) adding 207.142.131.236:80 to the address-pool
(trace) adding 207.142.131.245:80 to the address-pool
(trace) adding 207.142.131.246:80 to the address-pool
(trace) adding 207.142.131.247:80 to the address-pool
(trace) adding 207.142.131.248:80 to the address-pool
(trace) adding 207.142.131.202:80 to the address-pool
(trace) adding 207.142.131.203:80 to the address-pool

When I request http://127.0.0.1:1445/ the load-balancer takes the URL hashes it and sends it the one of the backends.

(trace) using hash-balancing: /wiki/Main_Page -> 207.142.131.204:80
(trace) using hash-balancing: /skins-1.5/monobook/main.css -> 207.142.131.204:80
(trace) using hash-balancing: /skins-1.5/common/commonPrint.css -> 207.142.131.213:80
(trace) using hash-balancing: /skins-1.5/common/wikibits.js -> 207.142.131.245:80
(trace) using hash-balancing: /w/index.php -> 207.142.131.206:80
(trace) using hash-balancing: /w/index.php -> 207.142.131.206:80
(trace) using hash-balancing: /w/index.php -> 207.142.131.206:80
(trace) using hash-balancing: /w/index.php -> 207.142.131.206:80
(trace) using hash-balancing: /skins-1.5/monobook/headbg.jpg -> 207.142.131.202:80
(trace) using hash-balancing: /skins-1.5/monobook/bullet.gif -> 207.142.131.245:80
(trace) using hash-balancing: /skins-1.5/common/images/poweredby_mediawiki_88x31.png -> 207.142.131.236:80
(trace) using hash-balancing: /images/wikimedia-button.png -> 207.142.131.203:80
(trace) using hash-balancing: /skins-1.5/monobook/bullet.gif -> 207.142.131.245:80
(trace) using hash-balancing: /skins-1.5/monobook/user.gif -> 207.142.131.202:80
(trace) using hash-balancing: /images/wiki-en.png -> 207.142.131.204:80

You see that the same URL results in the same address that is connected.

If one of the backends goes down, all requests that were meant for that backend are spread over the other backends. If you had 10 backends and 1 goes down, each backend has to server 1/9 URLs of the dead backend.

• you have 100 URLs, 10 backends
• each backend handles (100/10) = 10 URLs
• one backend goes down and its 10 URLs are spread over the other 9 backends
• each backend handles now its 10 URLs as before + (10/9) URLs of the dead backend

Hash balancing is following the ideas from http://icp.ircache.net/carp.txt

server.modules = ( ..., "mod_proxy_core", .. )

## works
$HTTP["url"] =~ "^/proxyme/" {
  proxy-core.balancer = "round-robin" 
  proxy-core.protocol = "http" 
  proxy-core.backends = ( "wikipedia.org" )

  ## to be done
  proxy-core.rewrite-response = (
    "Location" => ( 
      "^http://en.wikipedia.org/(.*)" => "http://127.0.0.1:1025/$1" ) )
  proxy-core.rewrite-request = (
    "URI" => (
      "^/proxyme(/.*)" => "$1" ) )
}

The above config forwards everything from /proxyme/ to wikipeedia.org. It resolves the domain wikipedia.org and is using one of the address to forward the request. As you already see there will be a request and response header rewriter for rewriting Location headers and URIs.

What works ?

• Round-Robin Load-Balancing
• Failover of a backend is down
• HTTP proxying
• HTTP/1.1
• Keep-Alive

What is missing

• FastCGI, SCGI and CGI backends
• Header Rewriting

Show me the source, Luke

In the source-browsers you can also take a look at the code to see what is going on in there.