Running the pipeline for the silver layer

In production, the curation pipeline will be invoked automatically after the ingestion pipeline completes successfully. Since we are still in the unit testing phase, we will trigger it manually using the following steps:

1. To invoke the pipeline for the silver layer, invoke the following commands on the Azure Cloud Shell. This pipeline run is counted as the first run of the pipeline. Notice we are sending ADLS_FOLDER=2021/04/28/18 as a parameter to the pipeline. You may recall that this was the very first folder in which historical data was saved in the bronze layer:

   RESOURCEGROUPNAME="training_rg"
   DATAFACTORYNAME="traininglakehousedf"
   PIPELINENAME="electroniz_curation_pipeline"
   az datafactory pipeline create-run --factory-name $DATAFACTORYNAME --name $PIPELINENAME --resource-group $RESOURCEGROUPNAME \
   --parameters "{\"STORAGE_ACCOUNT\":\"traininglakehouse\",\"BRONZE_LAYER_NAMESPACE...