Workflows Systems

32

open-source workflow systems

22 workflow systems provide WCI metadata

Contribute

Parsl

Productive parallel programming in Python

Python — 1.2.0 — 25 Mar 2025 — WCI metadata

Snakemake

This is the development home of the workflow management system Snakemake. For general information, see

Python — v9.1.1 — 25 Mar 2025 — WCI metadata

Covalent

Distributed workflows for quantum and HPC

Python — v0.238.0 — 25 Mar 2025 — WCI metadata

Cylc

A general purpose workflow engine with a particular gift for cycling.

Python — cylc-flow-8.4.1 — 25 Mar 2025 — WCI metadata

Nextflow

A DSL for data-driven computational pipelines

Groovy — Version 25.02.1-edge — 25 Mar 2025 — WCI metadata

AiiDA

A workflow manager for computational science with a strong focus on provenance, performance and extensibility.

Python — AiiDA v2.6.3 — 24 Mar 2025 — WCI metadata

SmartSim

Machine learning workflows with HPC applications (Python, C++, C, and Fortran)

Python — 0.8.0 — 24 Mar 2025 — WCI metadata

pyiron

A workflow manager for scientific computing on high performance computing infrastructures

python — pyiron_base-0.11.8 — 24 Mar 2025 — WCI metadata

SciPipe

Robust, flexible and resource-efficient pipelines using Go and the commandline.

Go — v0.12.0: Important bugfix. Minor API change. — 22 Mar 2025 — WCI metadata

Common Workflow Language

Interoperable workflow execution of containerized command line tools

YAML — 1.2.0 — 22 Mar 2025 — WCI metadata

DagOnStar

DAGonStar is a simple Python-based workflow engine that runs jobs on everything from the local machine to distributed virtual HPC clusters hosted in private and public clouds.

Python — v1.0.2 — 22 Mar 2025 — WCI metadata

nipype

Neuroimaging in Python: Pipelines and Interfaces

Python3 — 1.10.0 — 21 Mar 2025 — WCI metadata

StreamFlow

Towards Cloud-HPC Continuum

Python — 0.2.0.dev12 — 20 Mar 2025 — WCI metadata

WATTS

Workflow and Template Toolkit for Simulation

Python — 0.5.1 — 19 Mar 2025 — WCI metadata

libEnsemble

Tool for running dynamic ensembles.

Python — 1.4.3 — 19 Mar 2025 — WCI metadata

Makeflow

Makeflow is a workflow system for parallel and distributed computing that uses a language very similar to Make.

Make — release/7.14.8 — 18 Mar 2025 — WCI metadata

TaskVine

An execution system for large scale data intensive dynamic workflows.

Python — release/7.14.8 — 18 Mar 2025 — WCI metadata

Geoweaver

a lightweight workflow software to easily orchestrate pipelines from Python and shell scripts and preserve history of every execution

JavaScript — v1.9.0 — 12 Mar 2025 — WCI metadata

PyCOMPSs/COMPSs

Easy task-based parallelization and efficient execution in distributed infrastructures.

['Python', 'Java', 'C/C++'] — 3.3.1 Orchid Update — 06 Mar 2025 — WCI metadata

Merlin

Enabling Machine Learning HPC Workflows

Python — Version 1.12.2 — 04 Mar 2025 — WCI metadata

Maestro Workflow Conductor

Orchestrate your HPC workflows with ease

Python — v1.1.10 — 13 Feb 2025 — WCI metadata

Apollo Runtime System

A runtime system for the orchestration of workflows across the cloud-edge-Iot continuum.

Polyglot for Functions, AFCL for workflows — v1.1.0 — 13 Dec 2021 — WCI metadata

galaxy

Data intensive science for everyone.

Python — v24.2.3 — 26 Mar 2025

dask

Parallel computing with task scheduling

Python — 2025.3.0 — 25 Mar 2025

mlflow

Open source platform for the machine learning lifecycle

Python — MLflow 2.21.1 — 25 Mar 2025

pegasus

Pegasus Workflow Management System - Automate, recover, and debug scientific computations.

Java — 24 Mar 2025

cromwell

Scientific workflow engine designed for simplicity & scalability. Trivially transition between one off use cases to massive scale production environments

Scala — 88 — 24 Mar 2025

fireworks

The Fireworks Workflow Management Repo.

Python — v2.0.4 — 19 Mar 2025

jobflow

jobflow is a library for writing computational workflows.

Python — v0.1.19 — 17 Mar 2025

radical.entk

The RADICAL Ensemble Toolkit

Python — v1.42.0 — 05 Mar 2025

signac-flow

Workflow management for signac-managed data spaces.

Python — v0.29.0 — 03 Mar 2025

swift-t

Swift/T: High Performance Parallel Scripting Language

Java — 24 Feb 2025

For a comprehensive lists of all known computational workflow systems, see Computational Data Analysis Workflow Systems maintained by the CWL community.