Skip to main content

Argo

info

Please be aware that Argo-Workflows support is currently in progress and will soon become a mandatory requirement.

Argo Workflows

Argo Workflows is an optional component used in certain Gen3 commons to define and execute automated workflows.

It plays a role in simplifying and automating complex tasks within the Gen3 ecosystem.

Key Features of Argo Workflows​

Argo Workflows provides the following key features:

  • Workflow Definitions in YAML: Argo allows you to define workflows using YAML, providing a human-readable and easily maintainable way to describe your processes.

  • DAG-based Orchestration: The orchestration of tasks in Argo Workflows is based on directed acyclic graphs (DAGs), making it suitable for managing complex dependencies between tasks.

  • User Interface (UI) and API: Argo offers a user-friendly web-based UI and a programmable API to facilitate the execution of workflows on demand. This enables users to interact with and manage their workflows efficiently.

  • Metrics and Logs: Argo Workflows provides comprehensive metrics and logs, offering insight into the status and performance of running workflows. This observability feature is crucial for monitoring and troubleshooting workflows.

Use Cases in Gen3​

Argo Workflows is integral to Gen3, serving various purposes, including:

  • ETL Pipelines: Gen3 utilizes Argo Workflows to automate Extract, Transform, Load (ETL) processes, enabling efficient data ingestion and transformation.

  • Usersync: The Usersync process, responsible for synchronizing user data, is orchestrated using Argo Workflows, ensuring user information remains up-to-date.

  • Administrative and Housekeeping Tasks: Argo Workflows is employed for automating administrative and housekeeping tasks within the Gen3 environment, such as data archiving, backup management, and resource scaling.

Prerequisite for Workflow Automation​

Argo Workflows is a critical prerequisite for any Gen3 commons aiming to leverage workflow automation. The integration of Argo Workflows with Gen3's authentication and authorization systems is facilitated through the argo-wrapper service. This integration ensures secure and controlled access to workflow execution capabilities within the Gen3 platform.

By configuring Argo Workflows and the associated argo-wrapper service, Gen3 commons can harness the power of workflow automation to streamline data processing, user management, and administrative tasks.

For detailed information on setting up Argo Workflows and configuring the argo-wrapper service within your Gen3 environment, please refer to the official Gen3 documentation and relevant installation guides.