Blue Matter: Strong Scaling of Molecular Dynamics on Blue Gene/L

Blue Matter [1,2,3] is a molecular simulation framework and application developed to support the scientific goals of IBM’s Blue Gene project [4], to serve as a platform for research into application programming patterns for massively parallel architectures, and to explore ways to exploit hardware features of the Blue Gene/L architecture. A major design goal for Blue Matter has been to achieve strong scaling of molecular dynamics for moderately sized systems (10,000 – 100,000 particles) to node counts corresponding to ratios of atoms per node of order one. This supports one of the aims of the scientific component of the project, to carry out simulations on a scale that allows meaningful comparisons with experimental data. Results on a 43,222 atom protein/lipid system obtained from early production use of prototype Blue Gene/L hardware were recently published in the Journal of the American Chemical Society [5], another on the same system in in press and larger scale studies including microsecond-scale simulations of solvated protein and membrane protein systems are currently underway.

The use of Blue Gene hardware to advance our understanding of biologically important processes has been an integral part of the Blue Gene mission from the beginning of the project [4]. As part of that strategy, we started an application effort to support the scientific goals of the project and to also act as a concrete test-bed for research into application development for massively parallel machines. The Blue Matter development effort has tracked the evolution of the machine architecture with the goal of exploiting hardware facilities on the target machine and enabling a systematic exploration of parallel decompositions for molecular dynamics. The most recent results of this exploration are described in this paper. As part of our efforts to exploit the current Blue Gene/L machine architecture [6] we have explored the following: (1) Use of the global collective network (2) Machine topology effects (3d-torus) [7] (3) Low level interfaces vs. MPI (4)Use of both processors on the BG/L compute chip.

The Blue Matter architecture requires infrastructure to support extensive regression and validation because of the aggressive and experimental nature of the computational platform we are targeting and because of its support for multiple force fields (the models and their parameters used for classical molecular simulation). The two main requirements are that the force field parameters be properly implemented, and that the integrator correctly measures the forces on each atom and makes the appropriate update of position and velocity for each time step. The JACS publication [5] was based on a 118ns NVE simulation of a membrane-bound protein, and the total and kinetic energy drift over that long period of simulation was negligible, indicating that there was consistently correct bookkeeping and integration of all the interaction forces.