Workflows allow you to compose and chain multiple tasks or tools in a reproducible, scalable manner. Workflows are defined as Directed Acyclic Graphs (DAG), where each task can contain one or more Docker containers. This allows you to run each task in the DAG on a single machine or on a different machines with their own dedicated resources. For example, you can create a Workflow that scrapes data from multiple sources, runs each task on a dedicated CPU machine and then passes this data to a different task that trains a model on the scraped data on a GPU machine.
Onepanel Workflows are similar to AWS Step Functions, but are Kubernetes-native, are defined by YAML and Docker images and each task in the Workflow can run on different machines. They can also be run on any cloud provider that supports Kubernetes.
See Environment variables for more information on how environment variables can be added to Workspaces.
Onepanel's Workflows are based on the excellent Argo Workflows project. Most functionality is the same, one exception is that Onepanel Workflows entrypoint should always be a DAG template and do not support Steps templates. We also inject certain fields and abstract Argo to provide additional functionality and to simplify Workflow creation.
See Reference section for more information on defining Workflow Templates
You can define reusable templates for Workflows. All Workflow Templates are defined as Directed Acyclic Graphs (DAG), here's an example of how a DAG template is defined: