To Know or Not to Know: On the Needed Amount of Management Information

It is well accepted that a better management of scarce network resources helps in providing better service. However, it is often argued that due to the high complexity of management, a much more cost effective way to assure performance is just to acquire more resources. In this paper, we address this fundamental tradeoff in a rigorous way by showing exactly how much effort should be invested into management to gain the maximal benefit.

To focus our research, we consider management aspects of load balancing in distributed systems, in which incoming tasks arrive is a Poisson stream to a collection of n identical servers. A Supermarket Model studied by Mitzenmacher in 2001, suggests that using only a small amount of information about the local queue lengths of the servers and some simple randomization in job assignment, may lead to a very efficient load balancing algorithm. However, it was assumed that the management information is obtained at no cost.

In this work, we extend the Supermarket Model by explicitly incorporating the management costs. This Extended Supermarket Model (ESM) allows us to formally study the intuitively obvious tradeoff between the usefulness of management information and the cost of obtaining it. The main result presented in this paper is that for each service request rate, there exists an optimal number of servers that should be monitored. An interesting corollary of this finding is that knowing more about the global state of the system through detailed monitoring may not be only useless, but also harmful for the total quality of service provided by the system.