StreamFlow

WORKFLOW SYSTEMS  > STREAMFLOW

StreamFlow

Towards Cloud-HPC Continuum

Last updated: 17 Jun 2025     |     Release: 0.2.0.dev12

The StreamFlow framework is a container-native Workflow Management System (WMS) written in Python 3 and based on the Common Workflow Language (CWL) open standard. It has been designed around two main principles: allow the execution of tasks in multi-container environments, in order to support concurrent execution of multiple communicating tasks in a multi-agent ecosystem, and relax the requirement of a single shared data space, in order to allow for hybrid workflow executions on top of multi-cloud or hybrid cloud/HPC infrastructures.


Install

Tutorial

Documentation

Python
13
Forks
56
Stargazers
LGPL-3.0
License

Execution Environment

User Interfaces
  • CommandLine
Resource Managers
  • Local
  • SSH
  • Kubernetes
  • Docker
  • Docker Compose
  • Singularity
  • SLURM
  • PBS
Transfer Protocols
  • SCP
  • WebSocket (Kubernetes)

Terminology
Terminology below follows the definitions established by the Workflows Community Terminology.

Characteristics
Flow
Task
Data
Iterative
Granularity
Sub-workflows
Executables
Coupling
Loose
Dynamicity
Branches
Domain
Agnostic
Composition
Description
Standard (CWL)
Abstraction
Abstract
Modularity
Hierarchical
Orchestration
Planning
Dynamic
Execution
Runner
Resource Manager
Data Management
Transport
File-based
Storage
Distributed
Metadata Capture
Anomaly Detection
Provenance
Extensions
Replicated Storage
Cloud Integration

 

StreamFlow
https://streamflow.di.unito.it
https://github.com/alpha-unito/streamflow

Contributors

dependabot[bot]GlassOfWhiskeyLanderOttokinowslyqvsoch