Workflows Systems

35

open-source workflow systems

22 workflow systems provide WCI metadata

Contribute

TaskVine

An execution system for large scale data intensive dynamic workflows.

Python — release/7.15.3 — 16 Jun 2025 — WCI metadata

Swift/T

An MPI-based workflow system

Built with C; programmed with the Swift/T language — 1.6.6 — 16 Jun 2025 — WCI metadata

Snakemake

This is the development home of the workflow management system Snakemake. For general information, see

Python — v9.6.0 — 16 Jun 2025 — WCI metadata

StreamFlow

Towards Cloud-HPC Continuum

Python — 0.2.0.dev12 — 16 Jun 2025 — WCI metadata

Parsl

Productive parallel programming in Python

Python — 1.2.0 — 16 Jun 2025 — WCI metadata

AiiDA

A workflow manager for computational science with a strong focus on provenance, performance and extensibility.

Python — AiiDA v2.6.4 — 16 Jun 2025 — WCI metadata

Cylc

A general purpose workflow engine with a particular gift for cycling.

Python — cylc-flow-8.4.2 — 16 Jun 2025 — WCI metadata

Makeflow

Makeflow is a workflow system for parallel and distributed computing that uses a language very similar to Make.

Make — release/7.15.3 — 16 Jun 2025 — WCI metadata

Nextflow

A DSL for data-driven computational pipelines

Groovy — Version 25.04.4 — 16 Jun 2025 — WCI metadata

SmartSim

Machine learning workflows with HPC applications (Python, C++, C, and Fortran)

Python — 0.8.0 — 15 Jun 2025 — WCI metadata

Merlin

Enabling Machine Learning HPC Workflows

Python — Version 1.13.0b1 — 13 Jun 2025 — WCI metadata

WATTS

Workflow and Template Toolkit for Simulation

Python — 0.5.2 — 13 Jun 2025 — WCI metadata

Maestro Workflow Conductor

Orchestrate your HPC workflows with ease

Python — v1.1.10 — 11 Jun 2025 — WCI metadata

SciPipe

Robust, flexible and resource-efficient pipelines using Go and the commandline.

Go — v0.12.0: Important bugfix. Minor API change. — 10 Jun 2025 — WCI metadata

Geoweaver

a lightweight workflow software to easily orchestrate pipelines from Python and shell scripts and preserve history of every execution

JavaScript — v2.0.6 — 10 Jun 2025 — WCI metadata

pyiron

A workflow manager for scientific computing on high performance computing infrastructures

python — pyiron_base-0.13.0 — 06 Jun 2025 — WCI metadata

Covalent

Distributed workflows for quantum and HPC

Python — v0.240.0 — 05 Jun 2025 — WCI metadata

PyCOMPSs/COMPSs

Easy task-based parallelization and efficient execution in distributed infrastructures.

Python, Java, C/C++ — 3.3.3 Orchid Update — 02 Jun 2025 — WCI metadata

DagOnStar

DAGonStar is a simple Python-based workflow engine that runs jobs on everything from the local machine to distributed virtual HPC clusters hosted in private and public clouds.

Python — v1.0.2 — 27 May 2025 — WCI metadata

nipype

Neuroimaging in Python: Pipelines and Interfaces

Python3 — 1.10.0 — 16 May 2025 — WCI metadata

libEnsemble

Tool for running dynamic ensembles.

Python — 1.5.0 — 08 May 2025 — WCI metadata

Apollo Runtime System

A runtime system for the orchestration of workflows across the cloud-edge-Iot continuum.

Polyglot for Functions, AFCL for workflows — v1.1.0 — 13 Dec 2021 — WCI metadata

mlflow

Open source platform for the machine learning lifecycle

Python — 3️⃣ MLflow 3 3️⃣ — 17 Jun 2025

jobflow

jobflow is a library for writing computational workflows.

Python — v0.2.0 — 17 Jun 2025

galaxy

Data intensive science for everyone.

Python — v24.2.3 — 16 Jun 2025

cromwell

Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments

Scala — 90 — 16 Jun 2025

dask

Parallel computing with task scheduling

Python — 2025.5.1 — 16 Jun 2025

globus-compute

Globus Compute: High Performance Function Serving for Science

Python — 3.8.0 — 13 Jun 2025

toil

A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.

Python — 8.2.0 — 13 Jun 2025

ewoks

Extensible Workflow System

Python — 1.2.0 — 12 Jun 2025

pegasus

Pegasus Workflow Management System - Automate, recover, and debug scientific computations.

Java — 5.1.1 — 12 Jun 2025

fireworks

The Fireworks Workflow Management Repo.

Python — v2.0.4 — 09 Jun 2025

hyperflow

HyperFlow: a scientific workflow engine

JavaScript — v1.7.2 — 31 May 2025

radical.entk

The RADICAL Ensemble Toolkit

Python — v1.42.0 — 28 May 2025

signac-flow

Workflow management for signac-managed data spaces.

Python — v0.29.0 — 08 May 2025

For a comprehensive lists of all known computational workflow systems, see Computational Data Analysis Workflow Systems maintained by the CWL community.