Modern Workflows for Continuum and Cross-Facility Computing
SC23 BoF
Tuesday - Nov 14, 2023
12:15pm-1:15pm MT
Room 503-504
Science discovery relies increasingly on workflows to coordinate large and complex scientific experiments, ranging from cloud-based data preprocessing pipelines to multi-facility instrument-to-edge-to-HPC computational workflows. Continuum and cross-facility workflows have become prominent in computational sciences. Continuum workflows represent analysis pipelines that require continuous computing access, while cross-facility workflows span multiple sites, encompassing experiments and computing facilities. Cross-facility workflows offer resiliency for real-time workflows and can be viewed as a solution for the needs of continuum workflows. It is also important to consider continuum and cross-facility computing from a data perspective, ensuring workflow systems can handle different representations and storage systems.
In the second edition of this BoF (Tuesday Nov 14, 2023, 12:15pm-1:15pm MT), we will convene to discuss challenges, opportunities, research directions, and future pathways for continuum and cross-facility workflows. The perspectives of scientists as users of workflow systems will also be included, with users providing insights into the challenges specific to their science domains. Topics for discussion will encompass coordination and cooperation among computing and experimental facilities, metadata tracking across different sites, co-design of experiment facilities, standardization of task descriptions, edge-to-cloud continuum and data exchange, adaptive compression techniques, and the definition of common terms, building blocks, and concepts in workflows.
This BoF will yield tangible outputs, including lightning talks, live-poll questions and responses, and a draft of an updated community roadmap. Participants will be encouraged to join the Workflows Community Initiative to facilitate networking and foster potential international collaborations beyond the conference.
Agenda
-
12:15pm-12:20pm — Opening
Remarks
Rafael Ferreira da Silva – Oak Ridge National Laboratory
Kyle Chard – University of Chicago
Olivier Terzo – ACROSS/LINKS
-
12:20pm-1:00pm — Lightning talks: current state-of-the-art in challenges and opportunities
related to continuum and cross-facility workflows
Workflows to Power Digital Twins for Science
Tom Gibbs, NVIDIADOE/ASCR Activities and Research Directions Relevant to Workflows
Hal Finkel, U.S. DOEWorkflows without (institutional) borders : The DOE Integrated Research Infrastructure program
Debbie Bard, NERSCTaming the orchestration of complex workflows: the EU ACROSS project perspective
Alberto Scionti, ACROSS/LINKSWorkflow models for heterogeneous distributed systems
Iacopo Colonnelli, Università di TorinoAutomatic Steering in Large Task-based Workflows
Jorge Ejarque, eFlows4HPC/BSCDriving Modern Workflows from the Data Plane
Fred Suter, ORNL -
1:00pm-1:15pm — Open discussions