You're reading from Cloud Scale Analytics with Azure Data Services

Product type Book

Published in Jul 2021

Publisher Packt

ISBN-13 9781800562936

Pages 520 pages

Edition 1st Edition

Languages

Concepts

Author (1):

Patrik Borosch

Preface

Section 1: Data Warehousing and Considerations Regarding Cloud Computing

Chapter 1: Balancing the Benefits of Data Lakes Over Data Warehouses

Chapter 2: Connecting Requirements and Technology

Section 2: The Storage Layer

Chapter 3: Understanding the Data Lake Storage Layer

Chapter 4: Understanding Synapse SQL Pools and SQL Options

Section 3: Cloud-Scale Data Integration and Data Transformation

Chapter 5: Integrating Data into Your Modern Data Warehouse

Chapter 6: Using Synapse Spark Pools

Chapter 7: Using Databricks Spark Clusters

Chapter 8: Streaming Data into Your MDWH

Chapter 9: Integrating Azure Cognitive Services and Machine Learning

Chapter 10: Loading the Presentation Layer

Section 4: Data Presentation, Dashboarding, and Distribution

Chapter 11: Developing and Maintaining the Presentation Layer

Chapter 12: Distributing Data

Chapter 13: Introducing Industry Data Models

Chapter 14: Establishing Data Governance

Other Books You May Enjoy

Setting up a Synapse Spark pool

Now, let's examine the basic steps to spin up a Synapse Spark pool in this section.

This task is very easy to handle in a Synapse workspace:

Please navigate to the Management pane and there, in the Analytics pools section, select Apache Spark pools.
In the Details pane, click + New. The configuration blade for a new Apache Spark pool is displayed:
Figure 6.1 – Create Apache Spark pool – The Basics blade
Here you will name your new Spark pool and configure the node size value, enable Autoscale, and set the lower and upper boundaries for the autoscaling feature, if enabled. The last row in this view shows the potential cost of the lowest and the highest autoscaling setting. Click Next: Additional settings.
In the upper area of the Additional settings blade, you can now configure Auto-pause and Number of minutes idle, which sets the amount of idle time that will elapse before the cluster pauses. In the Component...