So far, we haven't done anything new, in the sense that everything we did was on-premises. This part of the book will focus on cloud leveraging of SSIS packages.
Before ADF V2, the only way to achieve orchestration with SSIS was to schedule our SSIS load on an on-premises (or an Azure) virtual machine, and then schedule an ADF V1.0 pipeline every n amount of minutes. If the data was not available at a specific time, the next ADF run would take it. Or, we had to tell ADF to wait for it before processing the rest of its pipeline.
Also, with the advent of SSIS 2017, the scaling out of package execution had to be done on-premises. There are a couple of issues with it:
- Who is responsible for the data warehouse data different usage? The developers that create and maintain the packages are not necessarily aware of the cloud implications of their processes. The data might be used in systems other than the ones they had in their specifications, when they first developed...
