Summary
In this chapter, we introduced a few Spark SQL applications in the textual analysis space. Additionally, we provided detailed code examples, including building a data preprocessing pipeline, implementing sentiment analysis, using Naive Bayes classifier with n-grams, and implementing an LDA application to identify themes in a document corpus. Additionally, we worked through the details of implementing an example of machine learning.
In the next chapter, we will focus on use cases for using Spark SQL in deep learning applications. We will explore a few of the emerging deep learning libraries and present examples of implementing deep learning related applications.
