DATA ANALYSIS FOR IMPROVING HPC OPERATIONS AND RESEARCH

PIs at University of Basel:
Florina Ciorba (Mathematics and Computer Science)
Sabine Gless (Law)
Thierry Sengstag (sciCORE)

PIs at University of Freiburg:
Gerhard Schneider (RZ) and Dirk von Suchodoletz (RZ)

PIs in Strasbourg:
Nicolas Lachiche (iCUBE, University of Strasbourg) and Ahmed Samet (INSA, Strasbourg)

Funding agency: Eucor – The European Campus (www.eucor-uni.org)
Duration: 01.09.2018-29.02.2020
Project Summary

Analysis of data from high performance computing (HPC) systems to improve HPC operations and research is an unsolved problem. This project addresses the challenge of HPC data analysis in a reproducible and legal manner. The data originates in the HPC systems of the consortium: NEMO at University of Freiburg (NEMO-UniFR), sciCORE at University of Basel (sciCORE‑UniBas), and HPC at University of Strasbourg (HPC‑UniStra).

The goals of the project are to analyze the data collected at NEMO‑UniFR to improve their research and operations activities, and to offer monitoring, operational, and research insights to also improve the sciCORE-UniBas and HPC-UniStra activities. The proposed approach entails: monitoring of systems and applications; legal compliance via de-identification and anonymization; and data analysis. The methods used include: HPC monitoring, legal controlling, de‑identification, anonymization, data aggregation, data mining, and insight extraction.

The outcome will be solutions for improving the HPC operations and research of three Eucor HPC centers, and satisfy the data protection and privacy requirements. The solutions are significant and transferable to other Eucor member institutions at the benefit of no/minimal legislative inquiries and data management overhead.

Publications

F. M. Ciorba, “The importance and need for system monitoring and analysis in HPC operations and research”, in Proceedings of the 3rd Baden-Würtenberg HPC Symposium (bwHPC 2016), Heidelberg, Germany, October 12, 2016. [C32.bib] (non-peer reviewed)

Talks

“The value of and need for system monitoring in HPC operations and research”,
Speaker: Florina M. Ciorba
Invited talk in the 3rd Baden-Würtenberg HPC Symposium (online agenda), University of Heidelberg, Germany, October 12, 2016.

Posters

F. M. Ciorba, G. Schneider, D. von Suchodoletz, A. Cavelan, T. Sengstag, S. Gless, N. Lachiche, and A. Samet, “Data Analysis for Improving High Performance Computing Operations and Research”, presented at the 5th bwHPC Symposium (online), Freiburg, Germany, September 24, 2018. [P16.bib]