Random Read Speedup in Compression Enabled Storage Systems

Modern primary storage systems are increasingly supporting real time compression, often based on the LZ77 and/or Huffman algorithm. There is a tension in such systems between achieving compression effectiveness along with performance and scalability; best compression effectiveness is thought to require more data to be compressed in a unit, but decompressing those large units to extract small blocks cause higher latencies and consume much more system resource. We show that, starting from the zlib base, both goals can be achieved, resulting in zlib -compatible highly compressed code streams that support low latency extraction of small blocks.

By: Cornel Constantinescu, Joseph Glider, David Chambliss, Dilip Simha

Published in: RJ10508 in 2013

LIMITED DISTRIBUTION NOTICE:

This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.

rj10508.pdf

Questions about this service can be mailed to reports@us.ibm.com .