Workload Management in Cloud Computing using Meta-Schedulers

Cloud computing is an emerging paradigm in the area of distributed systems. While numerous information technology vendors aggressively position themselves in offering various cloud services, many computer technologists earnestly provide their definitions of cloud computing, and compare this paradigm to its predecessors such as grid computing and utility computing. In this paper, we argue and demonstrate that we can extend our meta-scheduling design from grid computing to achieve adaptive usage of resources provided by cloud computing providers. Moreover, our meta-scheduling design has the advantage in managing the cloud resources as a collection instead of individual entities such that the organizations consuming the cloud resources can determine its cloud-using policies and delegate the controlling of the workloads to be executed to meta-schedulers. Evaluations of two workloads from real traces, in terms of user-perspective delay time and system-perspective resource utilization, are presented using our metascheduling algorithms and pools of cloud resources in different sizes. An interesting result is that the interaction between grid workload, backfill scheduling, and job routing from an enterprise cluster to a cloud can achieve better performance than a dedicated cluster of the combined size of the enterprise and cloud systems.