New paper: Distributed Management and Processing of ALICE Monitoring Data with Onedata

Discover how Onedata supports distributed ALICE monitoring by streaming live data from MonALISA into S3-backed storage and enabling secure access across HPC, cloud, and MLOps environments for efficient scientific processing.

Conference paper
Authors: Michał Orzechowski, Bartosz Baliś, Łukasz Dutka, Jacek Kitowski
Published in: CHEP 2024 [online] : conference on Computing in High Energy and Nuclear Physics : October 19-25, 2024, [Kraków, Poland] : book of abstracts; p. 196-197; Full details & cite →

Summary

This paper presents an architecture for the distributed management and processing of monitoring data generated within the ALICE experiment. The proposed approach uses the Onedata platform as a data plane, enabling live streaming of monitoring data from MonALISA and its storage on S3-based resources. The data can be accessed through POSIX, S3, and REST interfaces from HPC, cloud, and external MLOps environments. Onedata enables data to be transparently transferred and cached close to computational tasks, supporting efficient processing in a hybrid infrastructure. The solution also provides metadata management, access control, data integrity, hierarchical organization, and long-term archiving, facilitating reliable management of distributed scientific datasets.