Overview of AI Assistance and Copilot Integration

Microsoft Fabric has introduced a wide range of generative artificial intelligence (AI) experiences, called Copilot, that help accelerate each step of the analytics journey by potentially increasing overall developer productivity. This chapter explores the Copilot experiences built into different Fabric workloads:

• What is Copilot in Fabric?
• Copilot in data engineering and data science
• Copilot in Data Factory
• Copilot in Power BI

By the end of the chapter, you will have learned about Copilot and how you can leverage it to unlock new insights into your data.

Before diving into details, let’s take a step back and look at some technical requirements to get started with using Copilot.

Technical requirements

As of this writing in January 2024, the Microsoft Fabric Copilot feature is in preview, and you need P1, F64, or higher capacity to use Copilot.

Note

A capacity is a distinct pool of computing resources allocated to Microsoft Fabric and assigned to one or more Fabric workspaces. This capacity is available in multiple tiers, can be scaled up and down, and can be paused and resumed when needed. You can learn more about it here: https://learn.microsoft.com/en-us/fabric/enterprise/licenses#capacity-license.

By default, this feature is disabled, and your Fabric tenant admin needs to enable it by going to Admin portal | Tenant settings and then, in the Copilot and Azure OpenAI Service section, toggle the slider to enable this feature, as shown in Figure 11.1. The important point to note, however, is that Copilot sends customer data such as prompts, augmented data with prompts, and AI outputs to Azure OpenAI Service for processing, where it is temporarily stored...

What is Copilot in Fabric?

With the native integration of Copilot, a generative AI service in Microsoft Fabric, you can elevate your team’s productivity by leveraging these AI-assisted capabilities. While Copilot in different Fabric workloads provides different capabilities, in general, it aims to enhance developers’ productivity by offering a newer and quicker way for data integration, data transformation, building and training machine learning models, generating insights, and creating visualizations and reports. We will explore some of these capabilities in the next few sections.

Please note that Copilot is based on a probabilistic model and is still evolving; hence, you need to consider these points before you decide to use it:

• The output might vary from one execution to another or over time given its probabilistic nature. Likewise, the output shown in the following examples might be different from what you see; however, the goal remains the same –...

Copilot in data engineering and data science

Copilot for data engineering and data science workloads provides features to analyze and visualize your data to give you data insights with different types of visualizations, and it offers code-generation capability to improve productivity for data transformation and building and training machine learning models.

Copilot offers context-based automatic code generation, intelligent code completion, code documentation, fixing coding issues, automating routine tasks, providing standard coding templates, and so on. Further, it also allows you to visualize and analyze your data, which either comes from lakehouse or Power BI datasets or the dataframe you have created in your session, quicker and intuitively.

You can use the Copilot chat panel as a user interface or chat magic commands in notebook cells when taking advantage of this AI assistant. For our demonstration, we will first start using chat panels and then look into a few examples...

Copilot in Data Factory

There is currently one Copilot experience for Data Factory, which can be found when building a Dataflow Gen2. Just like the data science experience covered earlier in this chapter, the Data Factory Copilot lets you use natural language to describe the data transformations that you would like to apply to a dataset.

First, you will need to go through the get-data experience to acquire data. Optionally, apply any transformations you may know how to use on your own. Next, launch Copilot by creating a new Dataflow Gen2 and clicking the Copilot button on the Home tab of the ribbon. At this point, you will be greeted with a message informing you that Copilot can help you transform data or explain how the data is being transformed.

We can perform transformations such as combining all the address fields, in this case, city, state province, and country into a single column, as shown in Figure 11.27:

Figure 11.27 – Performing data...

Copilot in Power BI

Power BI has several key components, including data transformation and data modeling, culminating in a visual report that end users will consume. The Copilot experience is centered around the visual storytelling and reporting aspects of Power BI. This materializes in three ways: report page creation, narrative generation, and improving Q&A.

Let’s look at each of these Copilot capabilities.

Creating reports with the Power BI Copilot

The most common use for Copilot with Power BI is likely to be for creating reports. There are two features that come together to build reports. The first analyzes the dataset to suggest content for your report by using table relationships and column names, while the second one helps you create intuitive reports quickly. Figure 11.30 shows an example where Copilot has suggested several report pages, each with a short description of what would be displayed:

Figure 11.30 – The Power BI...

Summary

Native integration of generative AI in Microsoft Fabric in the form of Copilot aims to accelerate the analytics journey by potentially increasing overall developers’ productivity. In this chapter, we learned what Copilot is and what the requirements are to use it in Fabric, and then we looked at different examples for accelerating data integration, data transformation, visualizations, and reports.

The important point to note, however, is that what we covered in this chapter is just the beginning and barely scratched the surface. As this technology is still evolving and underlying models maturing, it is expected to be a game changer in the near future.

Throughout all the chapters of this book, we introduced you to the world of Microsoft Fabric, which opens up tremendous opportunities for you to build an analytics system quickly and derive business insights in this era of AI. We hope you enjoyed reading it as much as we enjoyed writing it. We cannot wait to see the...