A Performance Model of In-Situ Techniques
Ju Y, Vidal N, Perez A, Gainaru A, Suter F, Markidis S, Schlatter P, Klaskyll S, Laure E (2025)
Publication Type: Conference contribution
Publication year: 2025
Publisher: Institute of Electrical and Electronics Engineers Inc.
Pages Range: 209-216
Conference Proceedings Title: Proceedings - 33rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2025
Event location: Turin
ISBN: 9798331524937
DOI: 10.1109/PDP66500.2025.00036
Abstract
The computational capacity of High-Performance Computing (HPC) systems increases continuously with the rapid development of central processing units (CPUs) and graphic processing units (GPUs), while the in-/output (IO) subsystem develops relatively slowly and storage capacity is also limited. Data-intensive applications, which are designed to leverage the high computational capacity of HPC resources, typically generate a considerable amount of data for post-processing visualizations and data analytics. The limited IO speed and storage space could lead to constraints in the actual performance of these applications and, therefore, scientific discovery. In-situ techniques, where data is visualized/analysed while still in memory rather than through disk, can contribute to alleviating these problems as they can reduce or even fully avoid data writing/reading through the IO subsystem to/from storage. However, the overall efficiency of insitu techniques crucially depends on the characteristics of both the in-situ tasks and the applications, and the resource distribution among them. Therefore, choosing the right in-situ approach (synchronous, asynchronous, or hybrid) and resource allocation is essential to minimize overhead and maximize the benefits of concurrent execution. In this paper, we present a performance model of in-situ techniques to find the most beneficial in-situ approach and the preferred resource configuration. We verify the high accuracy of our approach with over 6800 measurements and provide use cases with different applications.
Authors with CRIS profile
Involved external institutions
Rechenzentrum Garching der Max-Planck-Gesellschaft (MPCDF) / Max Planck Computing & Data Facility
Germany (DE)
Oak Ridge National Laboratory
United States (USA) (US)
Royal Institute of Technology / Kungliga Tekniska Högskolan (KTH)
Sweden (SE)
Germany (DE)
Oak Ridge National Laboratory
United States (USA) (US)
Royal Institute of Technology / Kungliga Tekniska Högskolan (KTH)
Sweden (SE)
How to cite
APA:
Ju, Y., Vidal, N., Perez, A., Gainaru, A., Suter, F., Markidis, S.,... Laure, E. (2025). A Performance Model of In-Situ Techniques. In Proceedings - 33rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2025 (pp. 209-216). Turin, IT: Institute of Electrical and Electronics Engineers Inc..
MLA:
Ju, Yi, et al. "A Performance Model of In-Situ Techniques." Proceedings of the 33rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2025, Turin Institute of Electrical and Electronics Engineers Inc., 2025. 209-216.
BibTeX: Download