Sponsors
Objectives
- O1. Enabling the efficient use of new Ad-Hoc file systems to increase application throughput of HPC and cloud systems by leveraging the performance advantage of fast, node-local storage technologies.
- O2. Providing data-aware HPC scheduling decisions, based on global monitoring, to reduce I/O interference via globally coordinated minimization of data transfers between storage tiers and in-transit/in-situ processing facilities.
- O3. Balancing computation and data transfers by providing elastic mechanisms to dynamically adjust the ratio between the allocations of compute and storage resources.
- O4. Creating hybrid storage systems suitable for HPC and Cloud technologies.
- O5. Evaluating the feasibility of the proposed solutions through relevant use cases.
- O6. To plan and coordinate project activities and to promote results dissemination to increase impact.
As said before, the main goal of this project is to create an active I/O stack that dynamically adjusts computation and storage requirements through the use of ad-hoc file systems, new scheduling of storage resources integrated with monitoring along all levels of hybrid storage hierarchies in HEC, and I/O replication and storage malleability techniques to cope with runtime changes of data-intensive applications. TODO:sub-goals:
Work packages
WP1. Scalable I/O techniques for Ad-Hoc file systems
O1. Enabling the efficient use of new Ad-Hoc file systems. to increase application throughput of HPC and cloud systems by leveraging the performance advantage of fast, node-local storage technologies.
WP2. Monitoring and Data Scheduling
O2. Providing data-aware HPC scheduling decisions, based on global monitoring, to reduce I/O interference via globally coordinated minimisation of data transfers between storage tiers and in-transit/in-situ processing facilities.
WP3. Malleability and replication techniques
O3. Obtain techniques for balancing computation and data transfers by providing elastic mechanisms to dynamically adjust the ratio between the allocations of compute and storage resources.
WP4. New integrated techniques for hybrid data-driven applications in HPC and cloud
O4. Creating hybrid storage systems suitable for HPC and Cloud technologies.
WP5. Integration, validation, and optimization
O5. Evaluating the feasibility of the proposed solutions through relevant use cases.
WP6. Coordination and scheduling of activities
O6. To plan and coordinate project activities and to promote results dissemination to increase impact.
Deliverables
- D1.1. Report on design and implementation of ad-hoc file systems
- D1.2. Full version of Ad-hoc file systems published
- D2.1. Report on new I/O monitoring and modelling techniques
- D2.2. Report on Data scheduling techniques
- D3.1. Report on new I/O malleability and replications algorithms
- D3.2. Report on the design ad-hoc malleable file systems
- D6.1. Overall project plan and Web page
- D6.2. Intermediate Project Report
- D6.3. Final report of the project and conclusions
WP1. Scalable I/O techniques for Ad-Hoc file systems
WP2. Monitoring and Data Scheduling
WP3. Malleability and replication techniques
WP4. New integrated techniques for hybrid data-driven applications in HPC and cloud
WP5. Integration, validation, and optimization
WP6. Coordination and scheduling of activities
Publications
- 2025
- Improving I/O performance in HPC environments using the Expand Ad-Hoc file system
,
Authors: Diego Camarmas-Alonso, Felix Garcia-Carballeira, Alejandro Calderón, Jesus Carretero,
Journal: The Journal of Supercomputing
Contact
-
Welcome to our contact page for the Expand Parallel File System Project!
We appreciate your interest in our work and look forward to hearing from you.
Whether you have a question, a suggestion, or just want to point something out, we invite you to get in touch with us.
You can reach us via email at xpn.arcos.inf.uc3m.es@gmail.com, and we will do our best to respond to your message as quickly as possible.
Thank you again for your interest in our Expand Parallel File System Project !