TaskVine

WORKFLOW SYSTEMS  > TASKVINE

TaskVine

An execution system for large scale data intensive dynamic workflows.

TaskVine is a system building large scale data intensive dynamic workflows that run on HPC clusters, GPU clusters, and commercial clouds. As tasks access external data sources and produce their own outputs, more and more data is pulled into local storage on workers. This data is used to accelerate future tasks and avoid re-computing exisiting results. Data gradually grows 'like a vine' through the cluster. TaskVine can serve as an execution for other workflow systems such as Parsl and Dask.

Terminology

Terminology below follows the definitions established by the Workflows Community Terminology.

Terminology

Terminology below follows the definitions established by the Workflows Community Terminology.