Regulating Concurrency vs. Regulating Rate, for CPU Overload Protection of Web Servers

We compare two techniques of preventing server overload by controlling dispatch of mostly short transactional requests at the edge of a system of servers: (1) a mechanism that imposes a limit on the number of active requests and (2) a mechanism that imposes a limit on the rate at which requests are dispatched. While either technique will work well for a workload with low variation, the story gets more complicated if the workload has extreme variation in service time. We suppose a mechanism/policy separation, in which the limits imposed by a dispatch mechanism are set periodically based on policy considerations and current traffic characteristics. Since the dispatch mechanisms follow simple parameterized rules, they can not make ideal decisions for each individual request even with optimal parameter setting. We study the sensitivity of these mechanisms to the variation among requests handled by each mechanism in steady load scenarios where limits are constant or change occasionally as a result of policy-driven automation. The evaluation criterion is the quality of the regulation of the CPU utilization. We find that when the limits are manually set to fixed values neither dispatch mechanism is clearly better than the other. However, when matching automation of the limits is added then the rate-based dispatch mechanism produces more accurate results. On the other hand, the occupancy-based dispatch mechanism produces results that underload less often.

By: Wolfgang Segmuller; Michael Spreitzer; Malgorzata Steinder; Asser N. Tantawi

Published in: RC25090 in 2010


This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.


Questions about this service can be mailed to .