lighty's life: COMET meets mod_mailbox

COMET meets mod_mailbox

jan — Mon, 27 Nov 2006 14:22:00 +0000

Some time ago we got a request on how to implement COMET with lighttpd. I responded with a idea about a mod_multiplex which would allow the let the client open a COMET-channel and give the backend the possibility to feed multiple channels at once with the client to poll for new data.

Basicly it would separate the HTTP Request-Response cycle from the underlying connection. HTTP would be used to open the connection and reopen it in case it went away, but otherwise it would be just a data-channel for your JavaScript/AJAX content we want to send to the client when WE (the content-provider) want.

Let’s quote my understanding of COMET again:

My idea on this is to decouple the request from the COMET stream. AFAI understand COMET it is a ‘one-receiver-multiple-senders’ concept. The channel (a HTTP-response) is kept-open while the server/app can send multiple responses even without browser interaction to the client.

This is what you see in a web-chat application or what is used to send out stock-quotes in real-time to a large group of users.

In the classic HTTP world you either have to poll every one few seconds for new data or you keep the connection open and let it stream you the data. Polling is not instant and generates load even if no data is available. Streaming binds FastCGI (or similar) backend to the connection which limits the number of parallel users.

Decoupling the Connection from the content

We want

a persistent connection between to server and client to minimize the setup costs (keep-alive) and have immediate responses
a way to send data from a backend to multiple connections
to run a backend only to generate content

As this doesn’t fit into the classic model, we have to break it a bit. Classic means:

1. server reads HTTP Request 2. server forwards the request to the backend and waits for its response 3. server sends HTTP Response to the client

As soon as the backends closes its connection to the server, the server will assume that there is no more data to transfer and waits for the next client-request.

That’s what we want to change. We want to decouple the backend from the client-connection. To take it one step further we want to implement something we already know every well: a mailbox.

Queuing messages in a mailbox

If you are at home when the postman rings twice ... hmm, let’s start again.

If you are at home when the postman delivers the a letter you can read it right away. He rings the bell, you say hello, take the letter, read it.

If you are working hard in the office, the postman will the deliver the letter too, just to your mailbox. Actually several companies deliver letters, packages, ... all to your mailbox. You’ll pick the up when you come home.

mod_mailbox

First we need a name for the mailbox. The client opens a mailbox on the server, the server ties the client connection to this mailbox and sends all the data it gets for this mailbox right away to the client.

If the client drop the connection it either re-establishes or the server will remove the mailbox after some time.

A backend is started as usual and can send its response to multiple mailboxes and doesn’t have to care about connections being up or down. It only delivers to mailboxes (queues) managed by the mod_mailbox.

In case the backend delivered new content into a mailbox while the connection was temporary closed it will be sent as soon as the client re-opens the mailbox.

Can I use it ?

Not yet. This is a idea how to implement COMET in a nice, performing fashion. On the way to 1.5.0 it will be implemented. Meanwhile I’m looking for comments on the idea and if it matches your needs for COMET and AJAX.

"COMET meets mod_mailbox" by Marty

Sun, 24 Dec 2006 17:51:30 +0000

Most of the web apps I work on would really benefit from the COMET model. I had a 'heated debate' with a collegue about a month ago regarding the performance cost of AJAX - having COMET ability in lighttpd would, without a doubt, let me win the argument ;)

"COMET meets mod_mailbox" by kathy

Thu, 14 Dec 2006 09:37:47 +0000

If this became a reality, even if only a beta or alpha, I will switch to lighty immediately :)

"COMET meets mod_mailbox" by zellster

Wed, 29 Nov 2006 00:45:31 +0000

Additional work on the Java side with continuations: http://weblogs.java.net/blog/jfarcand/archive/2006/07/the_grizzly_com.html http://weblogs.java.net/blog/jfarcand/archive/2006/10/writting_a_come.html http://docs.codehaus.org/display/JETTY/Continuations http://www.webtide.com/downloads/whitePaperAjaxJetty.html

"COMET meets mod_mailbox" by zellster

Wed, 29 Nov 2006 00:44:07 +0000

"COMET meets mod_mailbox" by zellster

Wed, 29 Nov 2006 00:43:45 +0000

"COMET meets mod_mailbox" by Greg Daniluk

Tue, 28 Nov 2006 13:51:24 +0000

I've already implemented similar module for my web app. It works quite well with 1.4.* code. I cannot say it works great, because I haven't tested it with i.e. 10000 concurrent connections yet. Everything is implemented is C for best performance. The whole architecture is much simpler than cometd. You just need lighttpd and load additional module, that's all. During implementation I've faced few problems. 1. When the connection is idle, a browser is waiting for data and lighttpd for content, the socket fd cannot be readable or writable state. Then the browser is disconnecting (a user leaves the page). But the socket fd will be only half-closed. The Linux kernel won't notify lighttpd process about that and lighttpd will keep half-closed socket fd. There is a need for periodic manual cleanup of these fds. That's my temporary solution. I guess that Linux kernel needs a patch - provide some SETSOCKOPT to change semantics of select or epoll so they notify a process about error when sock isn't readable or writable. 2. HTTP server keeps the state what might be a problem during restart, log rotate or design problem for some. No solution, have to live with that. 3. By default a browser will allow only 2 connections to a web page. When a user will open second or third window then it will just stall, the browser will wait for first connections finish. My module detects that case and redirects the browser to www1 or www2 address and so on. Of course www1, www2 addresses must lead to the same lighttpd instance. These are other problems but they are already solved. I am going to release my module for free in 5-6 weeks when it will be better tested. My only concern is lighttpd stability. Some RoR folks discourage now using this server. I guess that stable lighttpd for comet applications might be a real hit.

"COMET meets mod_mailbox" by Felix

Tue, 28 Nov 2006 12:45:27 +0000

great Jan - really awesome what you create here. Keep up the good work

"COMET meets mod_mailbox" by Jan Kneschke

Mon, 27 Nov 2006 23:59:08 +0000

Looks like Bayeux is already doing want I was thinking about, and the best part: Dojo already supports it.

"COMET meets mod_mailbox" by Fobax

Mon, 27 Nov 2006 22:09:59 +0000

How would such a system scale to multiple lighty/comet servers? We're looking into such a system, but we'd want to be able to support 50,000 persistent connections, which is likely more than one lighty instance can handle.

"COMET meets mod_mailbox" by Xantus

Mon, 27 Nov 2006 19:08:08 +0000

D'oh Sorry about the double post. The comment form makes it so easy to do it! :)

"COMET meets mod_mailbox" by Xantus

Mon, 27 Nov 2006 19:07:42 +0000

You should join us in development of cometd. Check out our site at cometd.com

"COMET meets mod_mailbox" by Xantus

Mon, 27 Nov 2006 19:07:28 +0000

You should join us in development of cometd. Check out our site at cometd.com

"COMET meets mod_mailbox" by Thijs van der Vossen

Mon, 27 Nov 2006 19:02:28 +0000

Yes, this is indeed how we want to do it. Very nice.

"COMET meets mod_mailbox" by Scott

Mon, 27 Nov 2006 17:12:05 +0000

Some additional things that will make these even easier (and more efficient) for backend development: support for mailbox groups.

For example, backend tells lighty, add these mailbox ids to group "chat". From then on, the backend can address messages to the group "chat" without having to explicitly specify all the mailbox ids. This means less overhead in backendlighty communication and less parsing lighty has to do.

A group will be created when the first mailbox id is added to it. The group will be removed when the last mailbox id is removed from it (either by the backend explicitly, or by lighty expiring them).

For extra points, a send-to-group-excluding-these-mailboxes feature would mean for something like a chat server, you can have all clients in a single group, and the client (via the backend) can easily send to the group without receiving the message itself. The client can provide local echo like IRC does.

Named groups might also mean the backend could be restarted and continue where it left off without having to keep track of all the mailbox ids, but that's quite a simplification and it would probably need the membership info for other uses anyway.