Home Publications Research Grants Inventions & Patents Awards Additional Research Activities Faculties & Institutions Research Areas

Optimized Communication Scheme for Massively Parallel Simulations of Polydisperse Particle Systems with Large Size Ratios

Eibl S, Preclik T, Rüde U (2017)

---

Publication Language: English

Publication Type: Conference contribution, Abstract of lecture

Publication year: 2017

Event location: Hannover

Abstract

The standard method to parallelize particle based simulations with a hard contact interaction model
and impulsive contact reactions[1] is to subdivide the simulation domain into subdomains. Each
subdomain is handled by a different process and particle related information is communicated to
neighbouring domains. This next neighbour communication shows perfect scalability even for a very
large number of processes[2]. However it imposes an upper limit for the particle size as information
can not propagate to the subdomain after the next one. Making the subdomains larger is not an option
as it limits the usage of supercomputers which require thousands of processes to run efficiently.
One way to overcome this problem is to introduce global particles which are known on every
process. This however requires all to all communication to keep the particles in sync which impedes
good scaling performance on large systems.
Here we propose a new communication scheme which can handle polydisperse particles of arbitrary
size ratios. The largest particles are allowed to overlap more than one subdomain and are not
restricted in size. The communication scheme is based on point to point communication and
completely avoids expensive collective operations. Only processes which really have to know about a
particle will participate in the communication corresponding to this particle. Aggressive message
aggregation is used to reduce the amount of messages sent. Also an information caching system is
applied to further reduce the number of messages.
We implemented the communication scheme into our rigid body physics engine module pe of the
waLBerla framework[3] and investigated the scaling behaviour on the Juqueen supercomputer in
Jülich. We achieved perfect weak scalability up to the full machine. For the strong scaling the
communication protocol converges to an all to all communication as large particles overlap more and
more subdomains. Therefore as soon as this process starts the parallel efficiency deteriorates.

REFERENCES
[1] D. E. Stewart, Rigid-body dynamics with friction and impact. SIAM review 42.1 (2000): 3-39.
[2] T. Preclik & U. Rüde, Ultrascale simulations of non-smooth granular dynamics., Computational
Particle Mechanics 2 (2015), Nr. 2, S. 173--196
[3] C. Godenschwager et al., A framework for hybrid parallel flow simulations with a trillion cells
in complex geometries, Proceedings of the International Conference on High Performance
Computing, Networking, Storage and Analysis, 2013, 35

Authors with CRIS profile

Related research project(s)

How to cite

APA:

Eibl, S., Preclik, T., & Rüde, U. (2017). Optimized Communication Scheme for Massively Parallel Simulations of Polydisperse Particle Systems with Large Size Ratios. Paper presentation at Particles 2017, Hannover, DE.

MLA:

Eibl, Sebastian, Tobias Preclik, and Ulrich Rüde. "Optimized Communication Scheme for Massively Parallel Simulations of Polydisperse Particle Systems with Large Size Ratios." Presented at Particles 2017, Hannover 2017.

BibTeX: Download