You're reading from Mastering Microsoft Power BI – Second Edition - Second Edition

Product typeBook

Published inJun 2022

PublisherPackt

ISBN-139781801811484

Edition2nd Edition

Concepts

Business Intelligence

Authors (2):

Gregory Deckler

Brett Powell

View More author details

Connecting to Sources and Transforming Data with M

This chapter follows the environment and data source preparation described in Chapter 2, Preparing Data Sources, by implementing Power Query (M) queries in a new Power BI Desktop file to retrieve the required fact and dimension tables. Power Query queries are written in a data transformation language commonly called “M” or can be generated via the Power Query Editor user interface. These queries access data sources and optionally apply data transformation logic to prep the tables for the Power BI data model.

Power Query (M) expressions are becoming ubiquitous throughout the entire Microsoft data platform. These expressions are used with dataflows, which are reusable by multiple Power BI datasets. They are also supported by Azure Data Factory (ADF) meaning that data mashup processes that begin in Power BI can be scaled up if necessary. Finally, M queries underpin dataflows within Dataverse, Microsoft’s operational...

Types of Power Query M queries

In Chapter 2, Preparing Data Sources, SQL views were created, data sources configured, and the Power BI Desktop environment options applied. With these tasks accomplished, the dataset designer can finally start developing the data retrieval queries and parameters of the dataset. Power Query (M) queries are the means by which data sources are connected to and data imported into Power BI datasets. M queries are necessary to connect to data sources such as SQL views and can also perform data transformation as required.

The Power Query M language is a functional coding language more formally called the Power Query Formula Language. M includes over 700 functions that are used to connect to data and perform transformations of that data. The lines of M code that connect to and transform data are called a query.

There are a number of different types of queries that serve different purposes, including:

Data source parameters
Staging...

Creating Power Query M queries

As mentioned, the M language is a functional programming language that includes over 700 functions. Similar to other programming languages, M has its own specific syntax, structure, operators, and data types that must be used when coding. Experienced Power Query (M) developers, for example, are very familiar with Lists, Records, and Tables and common use cases and M functions available for working with these specific types.

While a full exploration of the entire M language is beyond the scope of this book, there are a number of important subjects regarding M that we cover in the following sections as well as providing readers with examples of more complex M queries. For readers interested in fully understanding the M language, we refer them to the official Power Query M language specification found here: https://bit.ly/3vmFSyr.

We’ll first take a look at numeric data types in M.

Numeric data types

For structured data sources, such...

Power Query M query examples

As demonstrated in the examples thus far, the combination of a mature data warehouse and a layer of SQL view objects within this source may eliminate any need for further data transformations. However, Power BI dataset designers should still be familiar with the fundamentals of M queries and their most common use cases, as it’s often necessary to further extend and enhance source data.

The following sections demonstrate three common data transformation scenarios that can be implemented in M. Beyond retrieving the correct results, the M queries also generate SQL statements for execution by the source system via query folding, and comments are included for longer-term maintenance purposes.

If you’re new to M query development, you can create a blank query from the Other category of data source connectors available within the Get Data dialog.

Alternatively, you can duplicate an existing query via the right-click context menu of...

Dataflows

Simply stated, dataflows are Power Query M queries created in the Power BI service. For non-My Workspace workspaces, the Power BI service provides an interface nearly identical to the Power Query Editor in Power BI Desktop for creating and editing Power Query queries.

Dataflows access source systems in the same manner as Power Query queries created in Power Query Editor within Power BI Desktop. However, the data ingested from these queries for import mode dataflows is stored in Dataverse-compliant folders within an Azure Data Lake Storage Gen2 instance. Dataflows can also be created for DirectQuery access to source systems.

There are several key advantages to the use of dataflows within enterprise BI. Chief among these advantages is reusability. Once created, a dataflow can be used as a data source within multiple different Power BI Desktop files during dataset design. Power BI dataflows are an option when using the Get data feature in both Power BI Desktop and...

Power Query M editing tools

Similar to other languages and project types, code-editing tools are available to support the development, documentation, and version control of M queries.

In addition to the Advanced Editor within Power BI Desktop and the dataflows editing interface of the Power BI service, dataset designers can use Visual Studio or Visual Studio Code to author and manage the M queries for Power BI and other Microsoft projects. These tools include common development features, such as IntelliSense, syntax highlighting, and integrated source control.

We’ll explore the different tools supporting M query development in the following sections.

Advanced Editor

In Power BI Desktop and for dataflows in the Power BI service, the M code for each query can be accessed from the Advanced Editor window within the Power Query Editor.

With the Power Query Editor open, select a query of interest from the list of queries on the left and click on the Advanced...

Summary

In this chapter, we’ve covered many of the components of the data retrieval process used to support the dataset for this project as described in Chapter 1, Planning Power BI Projects. This included constructing a data access layer and retrieval process for a dataset and using M queries used to define and load the dimension and fact tables of the dataset.

In the next chapter, we’ll leverage the M queries and design techniques described in this chapter to create import and DirectQuery data models.

Join our community on Discord

Join our community’s Discord space for discussions with the author and other readers: https://discord.gg/q6BPbHEPXp

The rest of the chapter is locked

You have been reading a chapter from

Mastering Microsoft Power BI – Second Edition - Second Edition

Published in: Jun 2022Publisher: PacktISBN-13: 9781801811484

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

undefined

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at €14.99/month. Cancel anytime

Authors (2)

Gregory Deckler

Greg Deckler is Vice President of the Microsoft Practice at Fusion Alliance and has been a professional technology systems consultant for over 25 years. Internationally recognized as an expert in Power BI, Greg Deckler is a Microsoft MVP for Data Platform and a superuser within the Power BI community with over 100,000 messages read, more than 11,000 replies, over 2,300 answers, and more than 75 entries in the Quick Measures Gallery. Greg founded the Columbus Azure ML and Power BI User Group (CAMLPUG) and presents at numerous conferences and events, including SQL Saturday, DogFood, and the Dynamic Communities User Group/Power Platform Summit.
Read more about Gregory Deckler

Brett Powell

Brett Powell is the owner of and business intelligence consultant at Frontline Analytics LLC, a data and analytics research and consulting firm and Microsoft Power BI partner. He has worked with Power BI technologies since they were first introduced as the PowerPivot add-in for Excel 2010 and has been a Power BI architect and lead BI consultant for organizations across the retail, manufacturing, and financial services industries. Additionally, Brett has led Boston's Power BI User Group, delivered presentations at technology events such as Power BI World Tour, and maintains the popular Insight Quest Microsoft BI blog.
Read more about Brett Powell

Personalised recommendations for you

Based on your interests and search pattern

Et al.

Ever wonder why speech recognition systems don't understand the Scottish accent, or what would happen if an astronaut only ate mac 'n' cheese, or other spurious reflections you'd have at a bar? We did, then collated those deliberations into absurd research articles with fake figures and methodologies inspired by even more fictionally absurd studies.

BookAug 2023230 pages5

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages4

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages5

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages1

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

BookDec 2023360 pages5

Mastering Tableau 2023

This book is a comprehensive resource to mastering your Tableau skills and becoming a BI expert. As you progress, you will learn how to build advanced dashboards and improve your storytelling to derive key business insight, as well as make you well-versed with advanced functionalities of Tableau in the business intelligence domain.

BookAug 2023684 pages

Building AI Applications with ChatGPT APIs

This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.

BookSep 2023258 pages5

Building AI Applications with ChatGPT APIs

This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.

BookSep 2023258 pages2

Data Engineering with AWS

Embark on a journey to master data engineering pipelines on AWS! Our book offers a hands-on experience of AWS services for ingesting, transforming, and consuming data. Whether you're an absolute beginner or someone with basic data engineering experience, this guide is an indispensable resource.

BookOct 2023636 pages5

Modern Data Architecture on AWS

Every organization wants an agile, performant, and cost-effective data platform that meets all their current and future business needs. Purpose-built AWS analytics services and their features play a big part in building such a modern data platform. This book brings to you all the design and architectural patterns that’ll help you achieve this goal.

BookAug 2023420 pages5

Practical Guide to Applied Conformal Prediction in Python

Discover the power of Conformal Prediction with the "Practical Guide to Applied Conformal Prediction in Python." Master the latest techniques to quantify uncertainty in machine learning and computer vision models, and seamlessly apply them to your industry applications.

BookDec 2023240 pages

TinyML Cookbook

With over 70 project-based recipes, the TinyML Cookbook is a practical guide that will help you to get the most out of your microcontrollers. It provides a comprehensive understanding of the theoretical foundations while giving you hands-on experience training ML models for deployment on Arduino Nano 33 BLE Sense, Raspberry Pi Pico, and SparkFun RedBoard Artemis Nano microcontrollers.

BookNov 2023664 pages