On the Design of a Deep Computing Service Cloud

This paper presents a detailed view of a system called Deep Cloud, which enables the offering of High Performance Computing (HPC) capabilities as a service, and the design decisions and architectural considerations involved in building such a system. Our system straddles traditional Grid computing and emerging Infrastructure-as-a-service (IaaS) systems. Similar to Grid computing, our goal is to enable scalable scientific applications. However, we extend the key principles from IaaS to enable predictable resource allocation of HPC resources with a pay-as-you-go model. Our approach uses dynamic pricing to shape demand for HPC resources, while providing access to HPC resources for both interactive and batched scientific workloads. We also describe a novel approach to HPC resource catalog creation: the conversion of an HPC 3D topology into an easily calculable resource inventory that supports predictable reservation and scheduling. This enables a dynamic resource placement mechanism for reservation requests which we call Just-In-Time placement. Just-In-Time placement collectively performs resource placement to achieve optimal HPC resource utilization. Finally, we describe the need for a coherent, integrated user experience when interacting with the system.

By: Zon-Yin Shae, Hani Jamjoom, Huiming Qu, Mark Podlaseck, Yaoping Ruan, Anshul Sheopuri

Published in: RC24991 in 2010


This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.


Questions about this service can be mailed to reports@us.ibm.com .