How to use Dependencies in Azure Data Factory?

Azure Data Factory supports four types of dependencies between the activities that can be used to control the execution flow of the connected activities within the same pipeline:

• Success, or green, dependency ensures that the next activity will be executed only when the previous activity executed successfully.

• Failure, or red, dependency ensures that the next activity will be executed only when the previous activity failed.

• Completion, or blue, dependency ensures that the next activity will be executed only when the previous activity is executed, regardless if it failed or succeeded.

• Skipped, or grey, dependency ensures that the next activity will be executed only when the previous activity is skipped or not executed.

In this article, we will see how to use these dependencies to control the activities run.

Prerequisites

• Azure Subscription.
• Azure Data Factory.

Scheduling the Pipeline Execution

1. Open the Azure Portal using your Internet Browser and search for your Azure Data Factory service.

2. In the Azure Data Factory, click Launch Studio to open the Azure Data Factory Studio:

3. In the Azure Data Factory Studio, click in the pencil icon to start designing the pipeline:

4. In the Author window, click New pipeline option:

5. In this pipeline, we will add a Get Metadata activity that will check if the file exists in the source container and run a copy activity on Success and Wait for one second on completion. If the copy activity failed due to the file absence, another copy activity will run on Failure. If the On Failure copy activity skipped, a delete activity will be executed. You can replace this with any scenario that meets your requirements:

6. If we execute the pipeline with the source file that exists in the source container, you will see that On Success and On Completion activities will be executed. In addition, with On Failure copy activity skipped, the On Skip delete activity will be executed:

7. If we execute the pipeline while the source file is not there, you will see that the OnFailure copy activity and On completion activity will be executed as expected: