AiiDA
A workflow manager for computational science with a strong focus on provenance, performance and extensibility.
Find the right system for your workflows
Explore the open source landscape and discover systems with rich community metadata.
38
Open source systems
26
Provide WCI metadata
Systems with WCI metadata
These projects provide structured metadata for richer discovery.
Snakemake
This is the development home of the workflow management system Snakemake. For general information, see
StreamFlow
Towards Cloud-HPC Continuum
pyiron
A workflow manager for scientific computing on high performance computing infrastructures
TaskVine
An execution system for large scale data intensive dynamic workflows.
Cylc
A general purpose workflow engine with a particular gift for cycling.
Makeflow
Makeflow is a workflow system for parallel and distributed computing that uses a language very similar to Make.
Maestro Workflow Conductor
Orchestrate your HPC workflows with ease
nipype
Neuroimaging in Python: Pipelines and Interfaces
Horus Runtime
Reproducible scientific workflows, from your laptop to the HPC cluster.
Parsl
Productive parallel programming in Python
Swift/T
An MPI-based workflow system
Ewoks
Meta-workflow system for scientific data processing and expert systems
Geoweaver
a lightweight workflow software to easily orchestrate pipelines from Python and shell scripts and preserve history of every execution
DragonHPC
Accelerate distributed Python workflows
Covalent
Distributed workflows for quantum and HPC
SmartSim
Machine learning workflows with HPC applications (Python, C++, C, and Fortran)
RADICAL-AsyncFlow
High Performance Asynchronous Workflow Scripting Library
SciPipe
Robust, flexible and resource-efficient pipelines using Go and the commandline.
DagOnStar
DAGonStar is a simple Python-based workflow engine that runs jobs on everything from the local machine to distributed virtual HPC clusters hosted in private and public clouds.
WATTS
Workflow and Template Toolkit for Simulation
RADICAL-EnTK
Ensemble Toolkit
PyCOMPSs/COMPSs
Easy task-based parallelization and efficient execution in distributed infrastructures.
libEnsemble
Tool for running dynamic ensembles.
Merlin
Enabling Machine Learning HPC Workflows
Apollo Runtime System
A runtime system for the orchestration of workflows across the cloud-edge-Iot continuum.
More workflow systems
A growing list of open source systems shared by the community.
nextflow
A DSL for data-driven computational pipelines
dask
Parallel computing with task scheduling
galaxy
Data intensive science for everyone.
mlflow
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
cromwell
Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments
pegasus
Pegasus Workflow Management System - Automate, recover, and debug scientific computations.
globus-compute
Globus Compute: High Performance Function Serving for Science
toil
A WDL, CWL and Python API supporting easy-to-use workflow engine. It is scalable, efficient and cross-platform (Linux/macOS).
jobflow
jobflow is a library for writing computational workflows.
fireworks
The Fireworks Workflow Management Repo.
hyperflow
HyperFlow: a scientific workflow engine
signac-flow
Workflow management for signac-managed data spaces.
For a comprehensive list of workflow systems, see Computational Data Analysis Workflow Systems maintained by the CWL community.