Managing the Response Time for Multi-tiered Web Applications

We introduce an edge-based technique for providing differentiated service and simple overload protection in the case of an inhomogeneous workload and a partially overlappingdeployment, notably including a multi-tier structure. Previous work has shown how reverse proxies doing inexpensive operations, under guidance from a controller, can provide differentiated service and overload protection assuming single-tier homogenous or disjoint deployment and, in some techniques, homogenous workload. We use a model-based controller with an optimization objective. The added complexities of inhomogeneous workload and deployment are handled by two key ideas: (1) the proxies classify traffic and protect against overload at an appropriate granularity, and (2) the controller solves an optimization problem subject to bounds derived from the deployment and using coefficients that characterize the computational intensity of the traffic. The modeling in the controller includes quantitative reasoning about computational load and thus supports other functions besides control of the proxies. We present some experiments that demonstrate differential queuing to manage the response times of applications with different intensities and changing and partially overlapping deployment.