Overview

Parsl is a flexible and scalable parallel programming library for Python. Parsl augments Python with simple constructs for encoding parallelism. Developers annotate Python functions to specify opportunities for concurrent execution. These annotated functions, called apps, may represent pure Python functions or calls to external applications. Parsl further allows invocations of these apps, called tasks, to be connected by shared input/output data (e.g., Python objects or files) via which Parsl constructs a dynamic dependency graph of tasks to manage concurrent task execution where possible.

Install Get started quickly

Tutorial Learn the basics

Documentation Explore the docs

Repository activity

601 Stargazers 212 Forks

Execution environment

User Interfaces

• Python API

Resource Managers

• Slurm
• LSF
• PBS
• Flux
• GridEngine
• HTCondor
• AWS
• Google Cloud
• Kubernetes

Transfer Protocols

• Globus
• HTTP
• FTP
• rsync

Terminology Terminology below follows the definitions established by the Workflows Community Terminology.

Characteristics

Flow

Data

Granularity

Sub-workflows

Coupling

Loose

Dynamicity

Branches

Domain

Agnostic

Composition

Description

API

Abstraction

Abstract

Modularity

Hierarchical

Orchestration

Planning

Dynamic

Execution

Runner

Data Management

Transport

Streaming

File-based

Storage

Shared

Distributed

Metadata Capture

Anomaly Detection

Monitoring

Extensions

Replicated Storage

Dynamic Parallelization

Cloud/Grid Support