Scalable GPU communication with code generation on stencil applications

Risso JVT, Bauer M, Carvalho PR, Rüde U, Weingaertner D (2019)

Publication Type: Conference contribution

Publication year: 2019

Publisher: IEEE Computer Society

Book Volume: 2019-October

Pages Range: 88-95

Conference Proceedings Title: Proceedings - Symposium on Computer Architecture and High Performance Computing

Event location: Campo Grande BR

ISBN: 9781728141947

DOI: 10.1109/SBAC-PAD.2019.00026

Abstract

Clusters with GPUs are mainstream in HPC as shown by the last edition of the Top500 list, increasing the demand for GPU capable scientific computing software. Programming large scale GPU systems in an efficient and future-proof way present numerous challenges, such as optimizations for a variety of GPUs and interconnect hardware, hiding communication overhead with computation and efficient domain partitioning. We present an improvement to the CUDA-based communication of stencil applications in the WALBERLA framework, achieving scalability while supporting different GPUs and communication infrastructures. We utilize the lattice Boltzmann Method for fluid flows as a representative of stencil-based scientific computing and implement a communication hiding strategy that is capable of adjusting to a system's computing and communication capabilities. We compare the use of CUDAMemCopy with the use of customized pack/unpack kernels and show that packing achieves almost linear weak scaling behavior in the Santos Dumont supercomputer with up to 128 GPUs. We also show that the proposed approach is not sensitive to the direction of the domain partitioning, one of the biggest challenges when communicating 3D domains in GPU-based stencil simulations.

Authors with CRIS profile

Martin Bauer Lehrstuhl für Informatik 10 (Systemsimulation) (LSS) Ulrich Rüde Lehrstuhl für Informatik 10 (Systemsimulation) (LSS) Daniel Weingaertner Lehrstuhl für Informatik 10 (Systemsimulation) (LSS)

Involved external institutions

Universidade Federal do Paraná (UFPR) BR Brazil (BR)

How to cite

APA:

Risso, J.V.T., Bauer, M., Carvalho, P.R., Rüde, U., & Weingaertner, D. (2019). Scalable GPU communication with code generation on stencil applications. In Proceedings - Symposium on Computer Architecture and High Performance Computing (pp. 88-95). Campo Grande, BR: IEEE Computer Society.

MLA:

Risso, Joao Victor Tozatti, et al. "Scalable GPU communication with code generation on stencil applications." Proceedings of the 31st International Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2019, Campo Grande IEEE Computer Society, 2019. 88-95.

BibTeX: Download