Various research communities have independently arrived at stream processing as a programming model for high-performance and parallel computation, including digital signal processing, databases, operating systems, and complex event processing. Each of these communities has developed some of the same optimizations, but often with conflicting terminology and unstated assumptions. This paper presents a survey of optimizations for stream processing. It is aimed both at users who need to understand and guide the system’s optimizer, and at implementers who need to make engineering trade-offs. To consolidate terminology, this paper is organized as a catalog, in a style similar to catalogs of design patterns or refactorings. To make assumptions explicit and help with trade-offs, each optimization is presented with its safety constraints (when does it preserve correctness?) and a profitability experiment (when does it improve performance?). We hope that this survey will help future optimization inventors to stand on the shoulders of giants from not just their own community.
By: Martin Hirzel, Robert Soulé, Scott Schneider, Bugra Gedik, Robert Grimm
Published in: RC25215 in 2011
LIMITED DISTRIBUTION NOTICE:
This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.
Questions about this service can be mailed to reports@us.ibm.com .