Summary
In this chapter, we introduced streaming data applications. We provided a few examples of using Spark SQL DataFrame/Dataset APIs to build streaming applications. Additionally, we showed the use of Kafka in structured streaming applications. Finally, we presented an example of creating a receiver for a custom data source.
In the next chapter, we will shift our focus to using Spark SQL in machine learning applications. Specifically, we will explore key concepts of feature engineering and machine learning pipelines.