Process-Oriented Performance Engineering Service Infrastructure for Scientific Software at German HPC Centers (ProPE)
Third party funded individual grant
Acronym: ProPE
Start date : 01.01.2017
End date : 31.12.2019
Website: https://blogs.fau.de/prope/
Project details
Scientific Abstract
The ProPE project will deploy a prototype HPC user support
infrastructure as a distributed cross-site collaborative effort of several
tier-2/3 centers with complementing HPC expertise. Within ProPE
code optimizing and parallelization of scientific software is seen as a
structured, well-defined process with sustainable outcome. The
central component of ProPE is the improvement, process-based
implementation, and dissemination of a structured performance
engineering (PE) process. This PE process defines and drives code
optimization and parallelization as a target-oriented, structured
process. Application hot spots are identified first and then
optimized/parallelized in an iterative cycle: Starting with an analysis of
the algorithm, the code, and the target hardware a hypothesis of the
performance-limiting factors is proposed based on performance
patterns and models. Performance measurements validate or guide
the iterative adaption of the hypothesis. After validation of the
hardware bottleneck, appropriate code changes are deployed and the
PE cycle restarts. The level of detail of the PE process can be
adapted to the complexity of the underlying problem and the
experience of the HPC analyst. Currently this process is applied by
experts and at the prototype level. ProPE will formalize and document
the PE process and apply it to various scenarios (single core/node
optimization, distributed parallelization, IO-intensive problems).
Different abstraction levels of the PE process will be implemented and
disseminated to HPC analysts and application developers via user
support projects, teaching activities, and web documentation. The
integration of the PE process into modern IT infrastructure across
several centers with different HPC support expertise will be the
second project focus. All components of the PE process will be
coordinated and standardized across the partnering sites. This way
the complete HPC expertise within ProPE can be offered as coherent
service on a nationwide scale. Ongoing support projects can be
transferred easily between participating centers. In order to identify
low-performing applications, characterize application loads, and
quantify benefits of the PE activities at a system level, ProPE will
employ a system monitoring infrastructure for HPC clusters. This tool
will be tailored to the requirements of the PE process and designed
for easy deployment and usage at tier-2/3 centers. The associated
ProPE partners will ensure the embedding into the German HPC
infrastructure and provide basic PE expertise in terms of algorithmic
choices, perfectly complementing the code optimization and
parallelization efforts of ProPE.
