The past, present, and future of Data Engineering
The Data Engineering practice has been around since the early internet era in the 1990s. Going back to Chapter 1, Fundamentals of Data Engineering, in the past, data engineers were mostly ETL developers using specific tools. Most of these tools were proprietary tools and located on-premises. The term data engineers itself didn't exist; the more common terms used to be data modelers, database admin, and ETL developer (ETL references the proprietary ETL tool's name). Each of the ETL tools had the necessary expertise and best practices surrounding them.
Now, in the present, Data Engineering has evolved into a more mature and singular role. This means that the practice has a lot more common principles, concepts, and best practices. This is due to two reasons – the rapid improvement in the technologies supporting the practice and the fact that Data Engineering has become a critical and central role to organizations.
...
