HPC-oriented Canonical Workflows for Machine Learning Applications in Climate and Weather Prediction
Autor(en): | Mozaffari, Amirpasha Langguth, Michael Gong, Bing Ahring, Jessica Campos, Adrian Rojas Nieters, Pascal Escobar, Otoniel Jose Campos Wittenbrink, Martin Baumann, Peter Schultz, Martin G. |
Stichwörter: | Computer Science; Computer Science, Information Systems; Earth system sciences; FAIR; Machine learning; Reproducibility; Workflow | Erscheinungsdatum: | 2022 | Herausgeber: | MIT PRESS | Journal: | DATA INTELLIGENCE | Volumen: | 4 | Ausgabe: | 2 | Startseite: | 271 | Seitenende: | 285 | Zusammenfassung: | Machine learning (ML) applications in weather and climate are gaining momentum as big data and the immense increase in High-performance computing (HPC) power are paving the way. Ensuring FAIR data and reproducible ML practices are significant challenges for Earth system researchers. Even though the FAIR principle is well known to many scientists, research communities are slow to adopt them. Canonical Workflow Framework for Research (CWFR) provides a platform to ensure the FAIRness and reproducibility of these practices without overwhelming researchers. This conceptual paper envisions a holistic CWFR approach towards ML applications in weather and climate, focusing on HPC and big data. Specifically, we discuss Fair Digital Object (FDO) and Research Object (RO) in the DeepRain project to achieve granular reproducibility. DeepRain is a project that aims to improve precipitation forecast in Germany by using ML. Our concept envisages the raster datacube to provide data harmonization and fast and scalable data access. We suggest the Juypter notebook as a single reproducible experiment. In addition, we envision JuypterHub as a scalable and distributed central platform that connects all these elements and the HPC resources to the researchers via an easy-to-use graphical interface. |
DOI: | 10.1162/dint_a_00131 |
Zur Langanzeige
Seitenaufrufe
6
Letzte Woche
3
3
Letzter Monat
2
2
geprüft am 15.05.2024