Thursday, May 15, 2008

COMET: A Hack for the Broken Web

This post provides insight into some ingenious engineering at Facebook. Erlang is the perfect language to implement the server side of COMET's long-polling with all its simultaneous connections.

However, all of that and COMET itself, is only necessary because of the fact that the World Wide Web is unfinished! In its current state the Web is BROKEN!

Most of the clients out there are not network nodes. They are half-nodes. They can initiate connections out, but do not accept incoming connections (on the WWW port, 80). Unlike the good old telephone network where every phone can call every other phone, the Web is like a television network; you can request and receive their programming, but they are not interested in receiving your programming!

What does this mean for web applications? Well, a chat application is a perfect example. Each client wants real-time updates of the status of its buddies. But since a server can't really push data to a typical client, the client has to poll. Of course polling is so inefficient that brilliant engineers have had to come up with things like long-polling, COMET, hugely distributed web servers, and the like--all of which are still pretty inefficient.

Wouldn't it be nice if each client were a real node? It could PUT its status to each of its buddy nodes any time its status changed. It could POST new chat messages directly to the buddy node. The centralized server would have far fewer responsibilities, like maintaining your buddy list and the address of the node you're logged in at.

What would be necessary to get the Web finished? Among other things, IPV6 or some other addressing scheme to get all the nodes uniquely addressed; new firewall and security rules and protocols to keep the bad guys at bay, and renewed interest in using HTTP to its full potential (using REST of course).

No comments: