Wednesday, November 28, 2007

Comet: The futuristic view of web applications – A food for thought!

I hope that this word “Comet” must be a new one for most of you. Might be some of you have heard about it in the name of Reverse AJAX or Server Push or HTTP Streaming or LazyHTTP or Long-Lived-Http-Requests or what not. These all are different names given to variations of the same idea, trying to achieve the same goal one way or other.

Here I will try to explain “What Comet actually Is?” So, let’s start with the basics. HTTP protocol is based on the pull model i.e. the client sends a request to the server, the server processes that request and sends the response back to the client. This conventional way leads to serious bandwidth + network traffic + scalability issues for web applications that need to provide some real time information i.e. news ticker or stock quotes or a web based chat client kind of things over the web. Considering this pull model of HTTP, the web application simulates the real time updates of the news / stock quotes ticker by periodically sending AJAX requests using some Java Script timers or a similar approach. So in that case my application is generating periodic XMLHttp Requests irrespective of the fact whether there are any updates on the server or not, this leads to a substantial over head in terms of the network bandwidth, web server’s scalability and performance. This way of simulating the real time updates is call the Ping Model, in which you are continuously ping-ing the server to get any of the updated contents available on the server.

What if I don’t want to Ping the server for updates? What if I want the web server itself to notify the client whenever there is an update? Can we really do that? Is it possible over HTTP? Is it possible in ASP.NET + IIS? These are the questions which must be storming your mind by now.

What I can say about all this is that, there are several giants who are into it, like Google is achieving this task some how on it’s Google Talk web chat panel inside GMail web based interface. Meebo is also providing that server push kind of thing for its web chat platform. There are also several (experimental + commercial) implementations of Comet which are available out there.

Still I have serious doubts about the scalability any such solution. What we know is that HTTP is based on the pull model, then how can these implementations achieve a real server push over HTTP? What as per my feeling, they are doing is simulating the server push using Long-Lived-Http-Requests or the other option to achieve Server Push is to bypass HTTP and use the Raw TCP sockets, but that would lead many other concerns related with security, scalability etc.

Another way to achieve this server push on Web is using Sockets in Flash movies, these flash movies can be easily embedded in web pages that can be shown over HTTP, but in that case we need a Flash Communication Server. Thus even for that, there is some thing that’s required from the server to achieve this push kind of behavior in web applications to provide real time information on web pages.

So far, Microsoft is quite on this. Let’s see how things turn out from there end.

Below are a few links to the related articles and working samples to increase your urge to learn more and more about Comet.


m2b2rt said...

Nice overview. I am interested in the same subject and believe that there are some issues with IIS supporting it. That is the idea of keeping the connection open. See this thread ASP.NET Comet
However, Lightstreamer seems to get around it somehow. Someon needs to work on a Grizzly equivalent in .NET and, more importantly, in IIS.

Mark said...

I have a better solution than Comet applications. Use Flash, it contains the XMLSocket object which can create an open socket connection to a server-side application, which means your application would use an IP address and a port number like any FTP or chat application (but within your browser). Each connection may need to be on it’s own thread, but you aren’t held back by threads so you can have 1000+ concurrent connections. Flash will send javascript commands to your client-side web application when a response comes in from the server, and you don’t need to find a solution for sending commands back to flash from javascript to make a request to the server (which will not be supported in some browsers). Instead, you can send a javascript xml request (AJAX) to the server and can relay the request to the server-side application.