Development

In the previous chapter, we looked at the importance of good architecture. We understood various aspects of architecture, both from infrastructure and data perspective. The chapter focused on other important aspects as well, such as backup and recovery design. In this chapter, we will learn about the best practices in dashboard designing and the art of developing a quality and user-friendly dashboard, which will cater to all the business requirements and provide business users with an ability to analyze their data, along with flexibility.

Along with this, we will see how to execute the project based on the data architecture we built in the previous chapter. We will also look at various best practices while developing dashboards.

This chapter is going to be the heart of the book as it will take you through the actual development once we have the requirements from the...

Data extraction process

The process of fetching the data from various source databases for further transformation and utilization in development of dashboard is the Data Extraction Process for Qlik Sense projects.

Data extraction is the starting step towards building the Qlik Sense dashboards. This process involves fetching the data from one or more databases, and merging and transforming them in such a way that it can be visualized graphically and an analysis can be done on that.

Qlik Sense uses the Extract, Transform, and Load (ETL) process to prepare the data for dashboard usage. Let us have a detailed look at the ETL process.

Extract, Transform, and Load (ETL)

ETL is the process used to create a data mart and data warehouse...

Building Data Model

Before starting any development on Qlik Sense, it is very important to understand the concept of Data modeling. In layman language, you can say that Data modeling is nothing but the way in which different data tables are linked together to get the required analysis. There are various data modeling techniques available, each suited for a specific kind of requirement.

Data modeling is very important for any BI project, because in data modeling you explore the logical relationship between the various attributes of the business. It helps you to visualize the different aspects of the business and find correlation between them. Thus, modeling becomes an important part of BI projects, where business wants to see the correlation between the various attributes and their impact on each other.

Out of the many data modeling techniques available, the best suited technique...

Common challenges of Data modeling

When you start data model for your Qlik Sense application which involves linking tables together, you may come across common challenges which may make your data model behave unexpectedly.

Let us have a look at some of the challenges which you may come across.

Synthetic Keys

Qlik Sense developers are aware that whenever you have common field names in two or more tables, Qlik Sense links them together automatically. When you have one field which is common, then you don't have a problem; it is a clean linking between them. However, when you have two or more fields that are common, Qlik automatically generates the synthetic keys and synthetic table.

Every time you reload the script, and...

Script Management

Often it happens with us that whenever we start writing code, we just keep on writing without focusing on the maintenance of the script. This may happen because we are in a hurry to complete the project. But eventually, any haphazard code is difficult to maintain and the code written by one developer would be difficult to manage for another developer.

For example, when you do the coding yourself, you may think that you have done a great job by delivering everything on time, but when you revisit your code after six months to do some modification, it often happens that you forget what you had written and what logic had been applied.

So being a good developer, your focus should not only be on the delivery of value added analysis to your end user, but also on maintenance of the script. Unless you handle your script well, you cannot truly call yourself a good developer...

Best Practices in Data Modeling

In the previous sections, we saw what are the common challenges or pain points in data modeling and how to tackle them. With this knowledge set, you can easily start building the data models for Qlik Sense. But there are couple of things which you should know in order to develop the best data model for a Qlik Sense project, which is effective, robust, and flexible in nature.

To do this, there are few points which you should be aware of:

• modeling schema: In data modeling, we had seen that the Star schema is the best suited schema modeling for Qlik Sense projects. Following image shows the comparison of the schemas from Qlik Sense project perspective:

• Avoid joins: At times, there are cases when we need to get fields from different tables into a single table to achieve some calculations or business logic. To do this, the first option which comes...

Data modeling Validation

Data validation is one of the critical and important points which is neglected by most of the developers. We will look at this point in detail in the next chapter, but I want to discuss briefly about validation of the data model. While developing the data model with best practices, we sometimes mess with the code and make small mistakes, which may not be wrong syntactically but may be wrong from data perspective.

Thus, it becomes important to do a preliminary validation of the data model to make sure that after doing all the calculations and linkage between the tables, the data gets populated correctly, which further helps in avoiding data related queries after entire development is done.

There are couple of things which you should do for data modeling validation. Let us have a look at them.

...

Security Strategy

Now that we have connected various data sources and developed the data model, it is time to think about the security.

Qlik Sense provides a solid security mechanism which helps to restrict access to the Qlik Sense at various levels. Qlik Sense provides the security mainly at four levels:

• Network security: Provides a secure communication channel between the user and server
• Server security by providing access to Qlik resources only to authorized users through QMC
• Process security by providing role based access to make sure proper testing is done for development
• App security by providing the row and column level data reduction through section access

Let us understand the server security and how Qlik does authentication:

Authentication: Qlik Sense server uses inbuilt proxy and virtual proxy as a gate keeper to allow only the authenticated user to enter Qlik Sense...

Visualization Strategy

The next step in the project is visualization. Now that we are ready with data, we can start developing the visualization through which data can be analyzed by the end users. End users, except the super user, might not have visibility of how data has been prepared; they will be more interested in analyzing the data and finding the facts from the data.

Let us start understanding the importance of visualization, but before that, let's look at some influencing quotes:

Numbers have an important story to tell. They rely on you to give them a clear and convincing voice.―Stephen Few

The purpose of visualization is insight, not pictures.―Ben Shneiderman

The goal is to turn data into information, and information into insight.―Carly Fiorina

All the preceding quotes try to tell us that we should present our data in such a way that they tell...

Choices of Objects

One of the important reasons why most of the dashboards and reports fail in getting the users' attention is the wrong choice of objects to represent data. It is not that the business users always look out for jazzy dashboards, but what they look for is useful information helping them to get actionable insights. This is possible only with the help of proper representation of the data, using proper charts and graphs.

Jazzy dashboards do attract users, but if those dashboards fail to provide proper insight into the data, they would not be useful .

Let us have a look at the various objects available in Qlik Sense, along with best suited data for them:

• Bar Chart: One of the widely used graphical representations is bar chart. Bar charts are very easy to understand. They are used when you want to compare values side by side; for example, comparison of sales...

Use of Extensions

Extensions, as the name suggests, allow you to extend the power of Qlik Sense by creating your own charts and graphs using the Qlik Sense APIs. Assume that you want to show some analysis to your business users but it cannot be shown using the available visualization library. In such cases, Qlik Sense allows you to create your own charts and use them in Qlik Sense application.

Qlik Sense has exposed several APIs to extend the capabilities of Qlik Sense. One of such APIs is visualization API. These are the same APIs that are used by Qlik Sense to create its own inbuilt visualizations.

If you are well versed with web technology like HTML and scripting like JavaScript, then you can easily create your own extensions. You can also use third-party APIs and visualization libraries, like D3, Chart.js, Echarts, and so on.

Let us have a look at few extensions which will...

Visualization best practices

We learned in previous section that visualization is the actual output which end users will be seeing. Considering that visualization helps users get insights into their data it becomes important to build more intuitive visualizations considering the best practices.

Let us have a look at few of the good practices

• Understand the data: It is not possible to present data with meaningful insight if you don’t understand the data. You should be to make the business user get the best possible information from dashboard especially when there are multiple datasources. You should also know the nature of the data, like whether it is a historical data, a nominal data, or a ratio data. This will help you to understand the best visualization to choose and this enables business quick insights. Thus, it is always important to understand your data before transferring...

Story Telling

Traditionally, the data which is required by the senior management is prepared by someone who knows how the management expects the data. This data is presented to the management in meetings and through presentations.

The objective of such meetings is to analyze the data and find the answers to business problems. However, the objective is hardly met in the same meeting, because when the questions starts arising, after looking at the data, no one has answers to them because the actual data resides in excel.

To overcome such problems and make meetings more interactive, Qlik Sense has introduced a powerful feature called story telling. Story telling allows you to share the insights found from the dashboards to other people in similar way as a power-point presentation, but the advantage is it is always connected with the actual data. In other words, you can say story...

Summary

At the end of the chapter, let us summarize all things we learnt in this chapter. Being the heart of the book, this chapter tried to teach us many aspects of development, starting from getting the data from the source system to converting it into the data insight visualization for end users to consume it.

We started with understanding the data extraction process, where we learned about ETL (Extract, Transform, and Load). Within the ETL process, we focused mainly on the extraction process, where we learned about full load and incremental load. We saw what kind of data would need what kind of load strategy.

In incremental load, we saw how to handle three main scenarios, namely insert, update, and delete. We also saw the scripting and sample outputs for each one of them.

After the extraction phase, we started with data modeling and learned about dimensional modeling with...