What do you get with eBook?

Instant access to your Digital eBook purchase

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Buy Now

SQL Server 2017 Integration Services Cookbook

Chapter 1. SSIS Setup

In this chapter, we will cover the following recipes:

SQL Server 2016 download
Installing JRE for PolyBase
Installing SQL Server 2016
SQL Server Management Studio installation
SQL Server Data Tools installation
Test SQL Server connectivity

Installing JRE for PolyBase

Java Runtime Engine (JRE) is required for PolyBase installations. SQL Server PolyBase is the technology that allows data integration from other sources other than SQL Server tables. PolyBase is used to access data stored in Hadoop File System (HFS) or Windows Azure Storage Blob (WASB).

As you will see later in this book, SSIS can now interact with these types of storage natively but having PolyBase handy can save us valuable time in our ETL.

Getting ready

For this recipe you will need to have access to the internet and have administrative rights on your PC to install JRE.

How to do it...

To download JRE, follow this link: http://www.oracle.com/technetwork/java/javase/downloads/index.html . You will see the screen shown in the following screenshot:

This directs you to the Java SE Download at Oracle.

Click the download link in the JRE section as shown in the following screenshot:

You must accept the license agreement to be able to select a file to download. Select Accept License Agreement as indicated in the following screenshot:

Since SQL Server 2016 only exists in a 64-bit version, download the 64-bit JRE. The version of Java SE runtime environment might be different from the one show in the screenshot, which is the one available at the time this book was written:.

Once downloaded, launch the installer. Click on Run as shown in Edge browser. Otherwise, go to your Downloads folder and double-click on the file you just downloaded (jre-8U102-windows-x64.exe in our case); you will see the following window:

The Oracle JRE installation starts. Click on Install. The following screen appears. It indicates the progress of the JRE installation.

Once the installation is completed, click on Close to quit the installer:

You are now ready to proceed to install SQL Server 2016. We'll do that in the next section.

How it works...

Microsoft integrated PolyBase in SQL Server 2016 to connect almost natively to the Hadoop and NoSQL platforms. Here are the technologies it allows us to connect to:

HDFS (Hortonworks and Cloudera)
Azure Blob Storage

Since Hadoop is using Java technology, JRE is used to interact with its functionalities.

Installing SQL Server 2016

This section will go through the installation of SQL Server engine, which will host the database objects used throughout this book.

These are the features available for SQL Server setup:

Database engine: It is the core of SQL Server. It manages the various database objects such as tables, views, stored procedures, and so on.
Analysis services: It allows us to create a data semantic layer that eases data consumption by users.
Reporting services (native): It allow us to create various reports, paginated, mobile, and KPI's for data consumption.
Integration services: It is the purpose of this book, SQL Server data movement service.
Management tools: We'll talk about these in the next section.
SQL Server Data Tools: We'll talk about these in the next section.

Getting ready

This recipe assumes that you have downloaded SQL Server 2016 Developer Edition and you have installed Oracle JRE.

How to do it...

The first step is to open the ISO file that you downloaded from the Microsoft Visual Studio Dev Essentials website as described in the SQL Server 2016 download recipe. If you're using Windows 7, you'll need to extract the ISO file into a folder. Third-party file compression utilities such as WinRAR, WinZip, or 7-Zip (and there are many more) can handle ISO file decompression. The setup files will be uncompressed in the folder of your choice. In other versions of Windows such as Windows 8.1, Windows 10, or Windows Server 2012 and beyond, simply double-click on the ISO file that you have downloaded previously and a new drive will appear in Windows Explorer.
Double-click on the file named Setup.exe to start the SQL Server installation utility. The features we're going to install are as follows:
- New SQL Server stand-alone installation or adding features to an existing installation: This will install a local instance (service) of SQL Server on your PC
- SQL Server Management Tools: The tools used to create, query, and manage SQL Server objects
- Install SQL Server Data Tools: This contains Visual Studio templates to develop and deploy SQL Server databases, integration services packages, analysis service cubes, and reporting services

From the installation utility, select the New SQL Server stand-alone installation... option as shown in the following screenshot. A new SQL Server setup window opens.

The Product Key page allows us to specify an edition to install. Since we're going to use the free Developer Edition, click Next to go to the next page, as shown in the following screenshot:

Accept the license terms and click Next to go to the next page, as shown in the following screenshot:

In this step, the SQL Server setup will check for product updates and will integrate itself into Windows update checks that are done regularly on your machine. This step is not mandatory but it's better to use the latest code. Check Use Microsoft Update... and click Next, as shown in the following screenshot:

Some updates might be found during setup. You can get more information on these updates by clicking the link in the More Information column. Click Next to install the updates, as shown in the following screenshot:

This step simply checks to make sure that the latest version of SQL Server is installed. Click Next once the setup files are installed, as shown in the following screenshot:

SQL Server setup will check several rules to ensure that the computer where we want to install it is setup properly. You might get a warning due to Windows firewall rules. This tells you that the port (1433 by default) is not open and SQL Server won't be available from outside your PC. Don't worry about it. Since we'll be using SQL Server from our PC only, we do not need to open any ports for now. Click Next to advance to the feature selection page, as shown in the following screenshot:

Select all features checked in the preceding screenshot and click Next, as shown in the following screenshot:

Instance configuration allows to specify a name for the SQL Server service. This is done by selecting the Named instance radio button. Since we'll only use one SQL Server instance, leave Default instance selected and click Next, as shown in the following screenshot:

This page allows SQL Server to be part of a PolyBase scale out group. Since we're only setting up SQL Server PolyBase to be used by one instance, leave the default Use this SQL Server as standalone PolyBase-enabled instance and click Next, as shown in the following screenshot:

Now for server configuration. This step allows us to specify distinct or specific service accounts. Since we're installing SQL Server on a single development machine, we'll use the default accounts, as shown in the following screenshot:

Click on the Collation tab as highlighted on the preceding screenshot. The default collation used by SQL server is SQL Latin1_General_CP1_CI_AS. This is a legacy collation. The choice of the collation is important for character string columns. The latest (fewer bugs) collation is Latin1_General_100. The last characters CI and AS are for case-insensitive and accent-sensitive, respectively.
We'll change the collation defaults. Click Customize... at the end of SQL Latin1_General_CP1_CI_AI, as shown in the following screenshot:

As stated previously, we'll use the Windows collation designator Latin1_General_100. Uncheck Accent-sensitive. This allows SQL Server to sort character columns without using accentuated characters. For example, suppose that our application has a FirstName column and we have the following first names:
- Joel
- Joël

If we query SQL Server filtering on FirstName = 'Joel' with the Accent-sensitive collation option, we end up retrieving the value Joel only. If we do not select the Accent-sensitive collation option, we will get both values.
Click OK when done to return to the previous screen. We'll do the same for analysis services; click Customize... to customize analysis service collation, as shown in the following screenshot:

Again, choose Latin1_General_100 in Collation designator and uncheck the Accent-sensitive checkbox. Click OK to return to the previous screen. Click on Next, as shown in the following screenshot:

This will direct you to the following screen. For the database engine configuration, we'll use Mixed Mode to allow us to use SQL Server logins and Windows logins. The default authentication is Windows authentication mode, which is more secure than SQL Server authentication because it uses the Kerberos security protocol, password, and account lockout policies, and password expiration. Make sure you use strong passwords for SQL Server logins. By default, password policy, password expiration, and user must change password at next login are turned on also for SQL Server login. You should not disable the password policy and the password expiration. Select the Mixed Mode radio box and enter a password for the SA account. Click on Add Current User as shown in the screenshot to add your Windows account as an administrator of the instance. You'll have all rights on it. Click Next, which will direct you to the Analysis Services Configuration window, as shown in the following screenshot:

For analysis services configuration, the Server Mode we'll use is Tabular Mode and again click on Add Current User as shown in the following screenshot to add your Windows account as an administrator of the service. Click Next.

For the Reporting services configuration, leave the default values and click Next, as shown in the following screenshot:

We're finally ready to install. Click Install to start the installation process, as shown in the following screenshot:

The following screenshot shows the installation progress:

Once the installation is complete, you get the following screen:

We're done. We just installed SQL Server 2016! In the next section, we'll install SQL Server Management Studio.

Note

The number, the order, and the appearance of the setup screens change slightly with every version of SQL Server, or even with a service pack. If you encounter a new screen not mentioned here, just use the default settings and proceed with the installation.

SQL Server Data Tools installation

The last part of our SQL Server 2016 setup is to install SQL Server Data Tools. This will install a Visual Studio Shell that contains BI templates necessary for the following:

SQL Server integration services
SQL Server analysis services
SQL Server reporting services
Database object management

Getting ready

We'll use SSDT throughout this book to create, deploy, and maintain our SSIS packages and some databases.

How to do it...

From the SQL Server 2016 setup utility, click on SQL Server Data Tools (SSDT). This will open the Download SQL Server Data Tools (SSDT) download page in your browser as shown in the following screenshot:

Here, there are two choices:

Install SSDT only: This is the simplest scenario. It only installs SSDT and a development shell.

Install Visual Studio and SSDT: You choose this if you plan to use source control inside Visual Studio or when you want to implement different types of development (.NET, Python, and so on) such as SSIS/SSAS/SSRS development. Since we'll talk about custom components in this book, we'll install Visual Studio Community Edition. This version is free for individuals.

Click on the Download Visual Studio Community 2015 link to download the Visual Studio installer.
Once downloaded, click on Run to launch the Visual Studio installer.
Accept the default installation type and click Install to start the installation process. This will give you the following window:

Once the installation is completed, since we haven't installed SSDT, don't launch Visual Studio yet. Close this window. We'll install SQL Server Data Tools first. Return to the browser window and click Download SQL Server Data Tools for Visual Studio 2015 as shown in the following screenshot:

This will direct you to the SSDT download screen shown as follows:

From the browser page that opens, choose English (United States). The SSDT setup executable file download starts. Since it's a small file, it takes only a few seconds to download. Once the download completes, click Run or double-click on the newly downloaded file to start SSDT installation. Accept the defaults and click Next to proceed to the next step, as shown in the following screenshot:

Accept the license agreement by checking the I agree to the license terms and conditions and click Install, as shown in the following screenshot:

The SSDT installer will download the necessary files and proceed to the installation, as shown in the following screenshot:

Once the installation completes, you might have to restart your computer. If that's the case, restart it, as shown in the following screenshot. Once that's done, look for SQL Server Data Tools in your Start menu and launch it.

From the File menu, select New Project. Once the New Project window appears, you will see Business Intelligence in the project templates, as shown in the following screenshot:

Close SSDT; we're done with it for now.

Key benefits

Acquaint yourself with all the newly introduced features in SQL Server 2017 Integration Services

Program and extend your packages to enhance their functionality

This detailed, step-by-step guide covers everything you need to develop efficient data integration and data transformation solutions for your organization

Description

SQL Server Integration Services is a tool that facilitates data extraction, consolidation, and loading options (ETL), SQL Server coding enhancements, data warehousing, and customizations. With the help of the recipes in this book, you’ll gain complete hands-on experience of SSIS 2017 as well as the 2016 new features, design and development improvements including SCD, Tuning, and Customizations. At the start, you’ll learn to install and set up SSIS as well other SQL Server resources to make optimal use of this Business Intelligence tools. We’ll begin by taking you through the new features in SSIS 2016/2017 and implementing the necessary features to get a modern scalable ETL solution that fits the modern data warehouse. Through the course of chapters, you will learn how to design and build SSIS data warehouses packages using SQL Server Data Tools. Additionally, you’ll learn to develop SSIS packages designed to maintain a data warehouse using the Data Flow and other control flow tasks. You’ll also be demonstrated many recipes on cleansing data and how to get the end result after applying different transformations. Some real-world scenarios that you might face are also covered and how to handle various issues that you might face when designing your packages. At the end of this book, you’ll get to know all the key concepts to perform data integration and transformation. You’ll have explored on-premises Big Data integration processes to create a classic data warehouse, and will know how to extend the toolbox with custom tasks and transforms.

What you will learn

[*] Understand the key components of an ETL solution using SQL Server 2016-2017 Integration Services [*] Design the architecture of a modern ETL solution [*] Have a good knowledge of the new capabilities and features added to Integration Services [*] Implement ETL solutions using Integration Services for both on-premises and Azure data [*] Improve the performance and scalability of an ETL solution [*] Enhance the ETL solution using a custom framework [*] Be able to work on the ETL solution with many other developers and have common design paradigms or techniques [*] Effectively use scripting to solve complex data issues

What do you get with eBook?