The Implications of Live Notifications
There is a trend among web 2.0 websites to add live chat or notifications to their websites. I have had the opportunity to work with a couple of them – one specialises in a product that provides such live chats to websites; the other is creating notifications and IM-like behavior for users browsing their pages.
Facebook of course added live notifications – you can see which of your friends are online and send them messages. Google has Gtalk purely on the web. This post analyzes the problems with such notifications.
Structure of Live Notifications
A chat or notification feature typically requires polling the web server. The user's browser has to go back to the server to check for any messages. It also writes the user's messages for other people. Such polling is done using Ajax webservice calls so that the user does not notice it.
Take the example of GTalk (Google's online chat). There are several webservices in that application:
1. when user logs in there is a notification POST saved in their data store so that user's contacts can see him/her online
2. when user logs in there is a "GET" to get the user's contacts who are online
3. when user types a message for a contact it is POSTed to that contact
4. when the contact types a message, the user's client GETs new messages addressed to him/her.
5. when user logs out there is a notification POST so that his/her contacts do not see him online
6. the user can also change their online status - that is a POST
Of these, items 2 and 4 require the user's client to constantly poll the server at a system-configurable interval. If this interval is high (say 2 minutes), the user gets frustrated. If it is low, the server takes a huge hit.
Implications of Live Notifications
Adding such notifications to a website requires careful analysis. The reason is because of the constant polling and writing data, Live Notifications increase the workload of a public website.
In a website there may be hundreds of online users - they may be going through different pages. The number of online users at the same time is called simultaneous user count. But the webservers themselves may be taking a lower hit - since many of the users may be reading content and otherwise browsing through a page, the number of concurrent users will generally be lower than the number of simultaneous users. Therefore a website can afford to have an infrastructure that supports a certain number of concurrent users. A set of concurrent users translates to a set of threads executing or waiting at the same time in a web server.
But, if the website introduces Live Notifications, the usage pattern changes dramatically. The constant polling makes the number of concurrent users close to the number of simultaneous users. Thus even if the the number of users in that website has not changed, the servers may be in for a beating.
This may not happen in a desktop IM client like Yahoo! Messenger - desktop IMs can afford peer-to-peer communication and therefore will not affect servers to the extent that a web model does.
Solutions for Live Notifications
The bottleneck in a Live Notification feature is the database server. Since conversation histories have to be stored, and every contact around the world should be able to send messages, database writes are unavoidable. They go up heavily in such a scenario. But the real problem is database reads. These may cost a server and a normal running server may crash if you add Live Notifications.
The web server is usually stateless in such interactions and thus scaling by adding more would work.
There are generally different levels of investment needed for companies deploying such features.
1. If your online (simultaneous) user base is low (say 2000-5000 users online at a time), you may be able to get away with a stateless webserver talking to a dedicated database server.
2. If your online user base is in the tens of thousands, you cannot afford reads to hit the database. You would need a solution in the middle-tier like memcached. This would keep the user's messages in the middle-tier and enable polling to retrieve new messages from the memory instead of the database. You will need a set of servers talking to a memcached server. You will also need failover for the memcached server.
3. If your online userbase is in the hundreds of thousands, you may need clusters of servers in the middle-tier and a complex addressing scheme so that messages can be picked up from a server. This is heavy investment in infrastructure, maintenance and engineering.
If your user base is low, then consider sticking to solution 1.
Before directly stepping into solution 3 ask yourselves this question - is adding such notifications a critical feature needed by my website? It is a specialty area.
Facebook of course added live notifications – you can see which of your friends are online and send them messages. Google has Gtalk purely on the web. This post analyzes the problems with such notifications.
Structure of Live Notifications
A chat or notification feature typically requires polling the web server. The user's browser has to go back to the server to check for any messages. It also writes the user's messages for other people. Such polling is done using Ajax webservice calls so that the user does not notice it.
Take the example of GTalk (Google's online chat). There are several webservices in that application:
1. when user logs in there is a notification POST saved in their data store so that user's contacts can see him/her online
2. when user logs in there is a "GET" to get the user's contacts who are online
3. when user types a message for a contact it is POSTed to that contact
4. when the contact types a message, the user's client GETs new messages addressed to him/her.
5. when user logs out there is a notification POST so that his/her contacts do not see him online
6. the user can also change their online status - that is a POST
Of these, items 2 and 4 require the user's client to constantly poll the server at a system-configurable interval. If this interval is high (say 2 minutes), the user gets frustrated. If it is low, the server takes a huge hit.
Implications of Live Notifications
Adding such notifications to a website requires careful analysis. The reason is because of the constant polling and writing data, Live Notifications increase the workload of a public website.
In a website there may be hundreds of online users - they may be going through different pages. The number of online users at the same time is called simultaneous user count. But the webservers themselves may be taking a lower hit - since many of the users may be reading content and otherwise browsing through a page, the number of concurrent users will generally be lower than the number of simultaneous users. Therefore a website can afford to have an infrastructure that supports a certain number of concurrent users. A set of concurrent users translates to a set of threads executing or waiting at the same time in a web server.
But, if the website introduces Live Notifications, the usage pattern changes dramatically. The constant polling makes the number of concurrent users close to the number of simultaneous users. Thus even if the the number of users in that website has not changed, the servers may be in for a beating.
This may not happen in a desktop IM client like Yahoo! Messenger - desktop IMs can afford peer-to-peer communication and therefore will not affect servers to the extent that a web model does.
Solutions for Live Notifications
The bottleneck in a Live Notification feature is the database server. Since conversation histories have to be stored, and every contact around the world should be able to send messages, database writes are unavoidable. They go up heavily in such a scenario. But the real problem is database reads. These may cost a server and a normal running server may crash if you add Live Notifications.
The web server is usually stateless in such interactions and thus scaling by adding more would work.
There are generally different levels of investment needed for companies deploying such features.
1. If your online (simultaneous) user base is low (say 2000-5000 users online at a time), you may be able to get away with a stateless webserver talking to a dedicated database server.
2. If your online user base is in the tens of thousands, you cannot afford reads to hit the database. You would need a solution in the middle-tier like memcached. This would keep the user's messages in the middle-tier and enable polling to retrieve new messages from the memory instead of the database. You will need a set of servers talking to a memcached server. You will also need failover for the memcached server.
3. If your online userbase is in the hundreds of thousands, you may need clusters of servers in the middle-tier and a complex addressing scheme so that messages can be picked up from a server. This is heavy investment in infrastructure, maintenance and engineering.
If your user base is low, then consider sticking to solution 1.
Before directly stepping into solution 3 ask yourselves this question - is adding such notifications a critical feature needed by my website? It is a specialty area.
Labels: performance, Web 2.0
posted by Ramiah Ariya at
3:02 AM
1 Comments:
good one ram. Deep on the problem which is absolutely real. However more details in the solution and some visuals will make your post much more readable.
By
mukund, at 6:55 AM
Post a Comment
<< Home