Model-Based and Model-Free Approaches to Autonomic Resource Allocation

A major goal of autonomic computing is to dynamically allocate computational resources so as to continually optimize high-level policy objectives. A key challenge to achieving this goal is to accurately estimate the impact of resource-level changes on application performance with respect to a Service Level Ageement (SLA). We compare two methodologies for accomplishing this: (i) developing a queuing-theoretic performance model for an application, and fitting its parameters online based on current state; (ii) using model-free reinforcement learning of resource valuation estimates based on trial-and-error learning. We describe these approaches in the context of a distributed architecture in which servers are allocated amongst multiple applications with independent time-varying loads. Each application has a local utility function, based on SLA payments as a function of relevant performance metrics. The overall system goal is to maximize the sum of local utility functions. Individual applications use one of the above methodologies to estimate resource valuations, which are then used by a resource arbiter to compute optimal allocations. We present empirical data illustrating the practicality and effectiveness of both methods in a realistic data center prototype. We highlight important tradeoffs between the methods, and point out potential benefits of a hybrid approach combining both methods.