SpliCS - Optimized Cache Line Placement Based on Reuse and Access Latency

Memory access latencies are much larger than processor cycle times, and this gap has been increasing over time. Cache performance is critical to bridging this gap. However, caches cannot be both large and fast. The design of primary caches involves a trade-off between access time and miss rate. This work presents techniques to balance the access time and the miss rate using novel primary cache design and management. Our design called, Split Latency Cache System (SpliCS), partitions the cache into a small, fast portion (cache A) and a larger, but slower portion (cache B). The goal is to improve the effectiveness of the cache hierarchy by allowing faster access to more useful lines while still allowing for larger caches to be used. Our results using commercial applications show that SpliCS achieves up to 35% improvement in the finite cache adder (measured in CPI) relative to a traditional primary cache of similar configuration and a 2 cycle access time.