A Scalable and Highly Available Web Server Accelerator

We describe the design, implementation and performance of a scalable and highly available Web server accelerator which runs under an embedded operating system and improves Web server performance by caching data. A single cache node implemented on a uniprocessor 200 MHz PowerPC 604 can serve up to 5000 pages/second, a figure which is an order of magnitude higher than that which would be achieved by a high-performance Web server running under a conventional operating system such as Unix or NT. The memory of our accelerator scales linearly with the number of cache nodes and the throughput scales almost linearly with the number of cache nodes as long as the front-end in not a bottleneck. Requests are routed to cache nodes using a combination of content-based routing and techniques which don't examine contents. Content-based routing reduces cache node CPU cycles but can make the front-end router a bottleneck. When content-based routing is not used, a request for cached object may initially be sent to the wrong node. In order to reduce the likelihood of this occurring, hot objects are replicated on multiple cache nodes. The replication policy is integrated with cache replacement and load balancing. The cache replacement policy takes into account the fact that there might be different bottlenecks in the system at different times. Depending upon which resource is likely to become a bottleneck, a different cache replacement algorithm is applied.