ISC 2026 Data Streaming BoF

Birds of a Feather / ISC 2026 Data Streaming BoF

Real-Time Scientific Data Streaming to HPC Nodes: Challenges and Innovations V

ISC 2026 Data Streaming BoF

Date Tuesday - Jun 23, 2026
Time 5:15pm—6:15pm CEST
Room Hall G1 - 2nd Floor
Conference Conference logo

As High-Performance Computing (HPC) enters the Exascale era, the scientific community faces a critical data paradox: our ability to generate data via high-fidelity simulations and next-generation instruments vastly outpaces our ability to store it. In many domains—from climate modeling and cosmology to experimental physics—simulation outputs and sensor inputs now frequently reach the Petabyte range per run. The traditional "compute-store-analyze" paradigm, which relies on dumping massive datasets to parallel file systems for post-processing, has become an unsustainable bottleneck. It introduces unacceptable latency, wastes energy on data movement, and often renders high-frequency data analysis practically inaccessible.

This BoF discusses this challenge by exploring the architectural shift from file-based workflows to high-performance streaming. In this new paradigm, storage is treated not as a buffer for post processing once simulations are completed, but as a sink for final scientific results, while analysis, visualization, and reduction occur in-transit or in-situ.

The schedule will be split into two sections. First, we will have a session of community-provided lightning talks which is intended to captivate and inspire the audience and familiarize them with the challenge at hand. The second part will be a "Workflow Challenge Workshop" where participants will be divided into small groups representing different stakeholders (facility operators, users, software developers). Each group will receive a real streaming workflow scenario and work together to identify technical, policy, and infrastructure requirements for implementation. Groups will document their solutions using a shared digital whiteboard, allowing cross-group visibility and iteration.

Organizers

Bjoern Enders (NERSC)
Bjoern Enders (NERSC)
Rafael Ferreira da Silva (ORNL)
Rafael Ferreira da Silva (ORNL)
Alex Upton (CSCS)
Alex Upton (CSCS)

Supporters