Storlet Engine: Performing Computations in Cloud Storage

The emergence of the cloud storage as infrastructure for serving large amounts of data over the WAN suggests new storage/compute paradigms. We propose expanding the cloud storage from only storing data to directly producing value from the data by executing computational modules - storlets - close to where the data is stored. This paper describes the Storlet Engine, an engine to support computations in secure sandboxes within the cloud storage. We describe its architecture and characteristics as well as the programming model for storlets. A rules mechanism allows implicit storlet activation via predefined conditionals complementing the explicit storlet activation. The security model supports storlets multi-tenancy and various types of sandboxes that constrain the resources available for the storlet execution. We also provide a performance study of the Storlet Engine prototype for OpenStack Swift cloud storage.