Enables integration of multiple pipelines to work in sync. Orchestration enables to automate the execution of multiple inter-related tasks to achieve the outcome.
Various individual pipelines can be picked and ordered on the canvas in the sequence of their execution. These pipelines can be configured to run one after another or in parallel.
On the workflow orchestration canvas, you can coordinate multiple pipelines as well as a combination of pipelines and existing workflows.
While designing a workflow, several actions can be inserted before and after each step to validate and define specific triggers. These help in defining what rules/triggers should be fired when a condition is met. You can either choose a single action (e.g., sending an email) or multiple actions (e.g., sending an email and updating a database entry) at each step of execution.
While designing a workflow, several actions can be inserted before and after each step to validate and define specific triggers. These help in defining what rules/triggers should be fired when a condition is met. You can either choose a single action (e.g., sending an email) or multiple actions (e.g., sending an email and updating a database entry) at each step of execution.
You can create a collection of pipelines to reflect their relationships and dependencies. These interdependent pipelines can be orchestrated using trigger rules.
Complex dependency settings and rules can be specified for coordinating between various upstream and downstream pipelines.
Once defined, these workflows can then be scheduled for execution using the scheduling interface.
Complex scheduling rules can be configured easily on the UI that orchestrates the execution of these workflows.
Once defined, these workflows can then be scheduled for execution using the scheduling interface.
Complex scheduling rules can be configured easily on the UI that orchestrates the execution of these workflows.
blog Feb 17, 2023
ETL vs ELT: Which data integration practice is right for you
ETL and ELT are common data integration practices that are extensively used in data science and business intelligence. While leading…
