Indexing Legacy Data-Sets for Global Access and Processing in Multi-Cloud Environments

Michał Orzechowski, Michał Wrzeszcz, Bartosz Kryza, Łukasz Dutka, Renata G. Słota, Jacek Kitowski

Future Generation Computer Systems, vol. 148, p. 150-159, ISSN 0167-739X

Abstract

Global data access and processing for multi-cloud applications proves challenging where there is a need for efficient access to large pre-existing, legacy data-sets. To address this problem, we created an indexing subsystem, allowing us to index and gather the metadata of legacy data-sets. The gathered information is later incorporated into a multi-cloud data management system to achieve global integration of legacy storage systems containing large data-sets. The solution is based on metadata management, organization and periodic monitoring of legacy data-sets, which makes possible scheme-less co-existence of a legacy storage system and a multi-cloud data management system for managing the same data collections. The approach has been initially evaluated in a multi-cloud deployment scenario. A large data collection consisting of legacy cultural data-sets was exposed to public and commercial infrastructure for data access and processing.

Cite this article

@article{orzechowski_indexing_2023,
  title   = {Indexing Legacy Data-Sets for Global Access and Processing in Multi-Cloud Environments},
  author  = {Michał Orzechowski and Michał Wrzeszcz and Bartosz Kryza and Łukasz Dutka and Renata G. Słota and Jacek Kitowski},
  journal = {Future Generation Computer Systems},
  year    = {2023},
  volume  = {148},
  pages   = {150--159},
  issn    = {0167-739X},
  doi     = {10.1016/j.future.2023.05.024},
}

Michał Orzechowski, Michał Wrzeszcz, Bartosz Kryza, Łukasz Dutka, Renata G. Słota, Jacek Kitowski,
Indexing Legacy Data-Sets for Global Access and Processing in Multi-Cloud Environments,
Future Generation Computer Systems,
2023,
Volume 148,
Pages 150-159,
ISSN 0167-739X,
https://doi.org/10.1016/j.future.2023.05.024