Remote Direct Memory Access (RDMA) traditionally aimed at improving high-performance computing applications and storage-area networks because CPU cycles and memory bus load can be reduced significantly by applying the zero-copy and direct data-placement techniques offered by RDMA. With the advent of iWARP, the RDMA technology is now available to the ubiquitous TCP/IP infrastructure and, thus, becomes interesting and relevant for legacy applications. In this paper, we present the conversion of the distributed C/C++ compiler distcc from sockets to the RDMA interface, and investigate the implications of using asynchronous and one-sided versus two-sided communication, RDMA-style buffer management, and notification mechanisms. We have designed a distcc extension that takes full advantage of asynchronous one-sided RDMA operations. By introducing a connection manager, we improve the communication efficiency and increase the scalability of distcc. Our tests with the RDMA-enabled distcc use a software-only iWARP implementation called SoftRDMA, which offers RDMA communication using conventional Ethernet adapters. The ideas and approach presented in this paper can be generalized to a wide range of applications.
By: Philip Frey; Bernard Metzler; Fredy Neeser
Published in: RZ3764 in 2010
LIMITED DISTRIBUTION NOTICE:
This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.
Questions about this service can be mailed to reports@us.ibm.com .