Summary

This chapter took us into the world of Databricks. You provisioned a Databricks workspace and examined it. In the workspace, you created a new Spark cluster and learned how to manage it.

You created a Databricks notebook, ran it interactively, and saw how to visualize data in your notebook. You also saw how to create a batch job from your notebook and learned about other alternatives for running code as a batch in your environment.

In the section that followed, you learned about Databricks tables and we examined additional capabilities, such as using Delta Lake to manage your data in your environment.

We saw how to add additional functionality using third-party libraries and how to create dashboards from your data.

Finally, we examined security features, such as access controls and secrets, and learned about networking features and how to integrate with Azure Monitor.

There are many more topics related to Azure Databricks that would have exceeded the capacity...