Message Proxies for Efficient, Protected Communication on SMP Clusters

This research addresses the problem of providing efficient, protected communication in an SMP cluster. Traditional approaches use either custom hardware of system-calls. This paper analyzes a new approach that uses an idle SMP processor to run a message proxy, a communication process that provides protected access to the network. We implement a message proxy and measure in detail the resulting communication overheads between a pair of IBM Model G30 SMPs. We derive a performance model that shows that cache-miss latency within an SMP influences message proxy performance significantly. We compare the performance of message proxies with custom hardware and system calls. Simulations of a suite of ten parallel applications demonstrate that message proxies match the performance of custom hardware for three of the ten applications, and are between 10-30% slower for the other seven applications. A direct cache-update mechanism to reduce cache misses improves the performance of message proxies on communication-intensive programs by 7-25%. Using system-calls results in the poorest performance. We conclude that the message proxies provide a viable alternative to custom hardware for protected communication.