Workflows Systems

32

open-source workflow systems

22 workflow systems provide WCI metadata

Contribute

Covalent

Distributed workflows for quantum and HPC

Python — v0.238.0 — 16 Apr 2025 — WCI metadata

pyiron

A workflow manager for scientific computing on high performance computing infrastructures

python — pyiron_base-0.11.10 — 15 Apr 2025 — WCI metadata

AiiDA

A workflow manager for computational science with a strong focus on provenance, performance and extensibility.

Python — AiiDA v2.6.3 — 15 Apr 2025 — WCI metadata

Geoweaver

a lightweight workflow software to easily orchestrate pipelines from Python and shell scripts and preserve history of every execution

JavaScript — v2.0.2 — 15 Apr 2025 — WCI metadata

Nextflow

A DSL for data-driven computational pipelines

Groovy — Version 25.02.3-edge — 15 Apr 2025 — WCI metadata

Snakemake

This is the development home of the workflow management system Snakemake. For general information, see

Python — v9.2.1 — 14 Apr 2025 — WCI metadata

WATTS

Workflow and Template Toolkit for Simulation

Python — 0.5.2 — 14 Apr 2025 — WCI metadata

Cylc

A general purpose workflow engine with a particular gift for cycling.

Python — cylc-flow-8.4.2 — 14 Apr 2025 — WCI metadata

TaskVine

An execution system for large scale data intensive dynamic workflows.

Python — release/7.15.1 — 14 Apr 2025 — WCI metadata

Makeflow

Makeflow is a workflow system for parallel and distributed computing that uses a language very similar to Make.

Make — release/7.15.1 — 14 Apr 2025 — WCI metadata

nipype

Neuroimaging in Python: Pipelines and Interfaces

Python3 — 1.10.0 — 11 Apr 2025 — WCI metadata

libEnsemble

Tool for running dynamic ensembles.

Python — 1.5.0 — 10 Apr 2025 — WCI metadata

Merlin

Enabling Machine Learning HPC Workflows

Python — Version 1.12.2 — 10 Apr 2025 — WCI metadata

DagOnStar

DAGonStar is a simple Python-based workflow engine that runs jobs on everything from the local machine to distributed virtual HPC clusters hosted in private and public clouds.

Python — v1.0.2 — 10 Apr 2025 — WCI metadata

Parsl

Productive parallel programming in Python

Python — 1.2.0 — 09 Apr 2025 — WCI metadata

StreamFlow

Towards Cloud-HPC Continuum

Python — 0.2.0.dev12 — 08 Apr 2025 — WCI metadata

PyCOMPSs/COMPSs

Easy task-based parallelization and efficient execution in distributed infrastructures.

['Python', 'Java', 'C/C++'] — 3.3.1 Orchid Update — 04 Apr 2025 — WCI metadata

SciPipe

Robust, flexible and resource-efficient pipelines using Go and the commandline.

Go — v0.12.0: Important bugfix. Minor API change. — 01 Apr 2025 — WCI metadata

SmartSim

Machine learning workflows with HPC applications (Python, C++, C, and Fortran)

Python — 0.8.0 — 28 Mar 2025 — WCI metadata

Common Workflow Language

Interoperable workflow execution of containerized command line tools

YAML — 1.2.0 — 22 Mar 2025 — WCI metadata

Maestro Workflow Conductor

Orchestrate your HPC workflows with ease

Python — v1.1.10 — 13 Feb 2025 — WCI metadata

Apollo Runtime System

A runtime system for the orchestration of workflows across the cloud-edge-Iot continuum.

Polyglot for Functions, AFCL for workflows — v1.1.0 — 13 Dec 2021 — WCI metadata

fireworks

The Fireworks Workflow Management Repo.

Python — v2.0.4 — 15 Apr 2025

cromwell

Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments

Scala — 88 — 15 Apr 2025

mlflow

Open source platform for the machine learning lifecycle

Python — 15 Apr 2025

pegasus

Pegasus Workflow Management System - Automate, recover, and debug scientific computations.

Java — 15 Apr 2025

dask

Parallel computing with task scheduling

Python — 2025.3.0 — 15 Apr 2025

galaxy

Data intensive science for everyone.

Python — v24.2.3 — 15 Apr 2025

jobflow

jobflow is a library for writing computational workflows.

Python — v0.1.19 — 14 Apr 2025

swift-t

Swift/T: High Performance Parallel Scripting Language

Java — 10 Apr 2025

signac-flow

Workflow management for signac-managed data spaces.

Python — v0.29.0 — 10 Apr 2025

radical.entk

The RADICAL Ensemble Toolkit

Python — v1.42.0 — 03 Apr 2025

For a comprehensive lists of all known computational workflow systems, see Computational Data Analysis Workflow Systems maintained by the CWL community.