Before getting started with SAP Lumira, you need to learn about data discovery. Maybe this term is not new to you, however, we need to clarify what it is in the case of SAP Lumira. In addition, it will be interesting and useful to learn some of the theory.
In this chapter you will learn:
What data discovery is, and how it complements a traditional data warehouse (DWH) and business intelligence (BI)
Data discovery terms
Common organizational architecture and the role of data discovery in the organization
We meet with one of the most powerful and flexible data discovery tools – SAP Lumira
We meet Unicorn Fashion, an e-commerce retail company
We are living in a century of information technology. There are a lot of electronic devices around us which generate lots of data. For example, you can surf the Internet, visit a couple of news portals, order new Nike Air Max shoes from a web store, write a couple of messages to your friend, and chat on Facebook. Your every action produces data. We can multiply that action by the amount of people who have access to the internet or just use a cell phone, and we get really BIG DATA. Of course, you have a question: how big is it? Now, it starts from terabytes or even petabytes. The volume is not the only issue; moreover, we struggle with the variety of data. As a result, it is not enough to analyze only the structured data. We should dive deep in to unstructured data, such as machine data which are generated by various machines.
Nowadays, we should have a new core competence—dealing with big data—, because these vast data volumes won't be just stored, they need to be analysed and mined for information that management can use in order to make right business decisions. This helps to make the business more competitive and efficient.
Unfortunately, in modern organizations there are still many manual steps needed in order to get data and try to answer your business questions. You need the help of your IT guys, or need to wait until new data is available in your enterprise data warehouse. In addition, you are often working with an inflexible BI tool, which can only refresh a report or export it in to Excel. You definitely need a new approach, which gives you a competitive advantage, dramatically reduces errors, and accelerates business decisions.
So, we can highlight some of the key points for this kind of analytics:
Integrating data from heterogeneous systems
Giving more access to data
Using sophisticated analytics
Reducing manual coding
Simplifying processes
Reducing time to prepare data
Focusing on self-service
Leveraging powerful computing resources
We could continue this list with many other bullet points.
If you are a fan of traditional BI tools (later in this chapter, we will compare BI and data discovery tools), you may think that it is almost impossible. Yes, you are right, it is impossible. That's why we need to change the rules of the game. As the business world changes, you must change as well.
Maybe you have guessed what this means, but if not, I can help you. In this book, I will focus on a new approach of doing data analytics, which is more flexible and powerful. It is called data discovery. Of course, we need the right way in order to overcome all the challenges of the modern world. That's why we have chosen SAP Lumira—one of the most powerful data discovery tools in the modern market. But before diving deep into this amazing tool, let's consider some of the challenges of data discovery that are in our path, as well as data discovery advantages.
Let's imagine that you have several terabytes of data. Unfortunately, it is raw unstructured data. In order to get business insight from this data you have to spend a lot of time in order to prepare and clean the data. In addition, you are restricted by the capabilities of your machine. That's why a good data discovery tool usually is combined of software and hardware. As a result, this gives you more power for exploratory data analysis.
Let's imagine that this entire big data store is in Hadoop or any NoSQL data store. You have to at least be at good programmer in order to do analytics on this data. Here we can find other benefit of a good data discovery tool: it gives a powerful tool to business users, who are not as technical and maybe don't even know SQL.
Tip
Apache Hadoop is an open source software project that enables distributed processing of large data sets across clusters of commodity servers. It is designed to scale up from a single server to thousands of machines, with a very high degree of fault tolerance. Rather than relying on high-end hardware, the resilience of these clusters comes from the software's ability to detect and handle failures at the application layer.
A NoSQL data store is a next generation database, mostly addressing some of the following points: non-relational, distributed, open-source, and horizontally scalable.
You may be confused about data discovery and business intelligence technologies; it seems they are very close to each other or even BI tools can do all what data discovery can do. And why do we need a separate data discovery tool, such as, SAP Lumira?
In order to better understand the difference between the two technologies, you can look at the table below:
|
Enterprise BI |
Data discovery | |
|---|---|---|
|
Key users |
All users |
Advanced analysts |
|
Approach |
Vertically-oriented (top to bottom), semantic layers, requests to existing repositories |
Vertically-oriented (bottom-up), mushup, putting data in the selected repository |
|
Interface |
Reports, dashboards |
Visualization |
|
Users |
Reporting |
Analysis |
|
Implementation |
By IT consultants |
By business users |
Let's consider the pros and cons of data discovery:
Pros:
Rapidly analyze data with a short shelf life
Ideal for small teams
Best for tactical analysis
Great for answering on-off questions quickly
Cons:
As a result, it is clear that BI and data discovery handles their own tasks and complement each other.
Most organizations have a data warehouse. It was planned to supporting daily operations and to help make business decisions. But sometimes organizations need to meet new challenges. For example, Retail Company wants to improve their customer experience and decide to work closely with the customer database. Analysts try to segment customers into cohorts and try to analyse customer's behavior. They need to handle all customer data, which is quite big. In addition, they can use external data in order to learn more about their customers. If they start to use a corporate BI tool, every interaction, such as adding new a field or filter, can take 10-30 minutes. Another issue is adding a new field to an existing report. Usually, it is impossible without the help of IT staff, due to security or the complexities of the BI Enterprise solution. This is unacceptable in a modern business. Analysts want get an answer to their business questions immediately, and they prefer to visualize data because, as you know, human perception of visualization is much higher than text. In addition, these analysts may be independent from IT. They have their data discovery tool and they can connect to any data sources in the organization and check their crazy hypotheses.
There are hundreds of examples where BI and DWH is weak, and data discovery is strong.
Let's discuss the various possible users of the data discovery tool and their advantages. Here's a small schema of possible data flows in an organization:
There are many systems which generate data. Organizations try to catch all the data and put it in data warehouse. In addition, for big data volumes they can use Hadoop because it offers cheap storage, high scalability, distributed computing power, and can be used as a staging area for raw data.
On top of the DWH we have a business intelligence tool, such as SAP BusinessObjects, MicroStrategy, Oracle BI, or Cognos BI. This uses data from the DWH and gives the opportunity to build reports and visualize data. But it takes long a time to create the semantic layer via a BI tool such as the universe, schema or repository. In addition, it requires technical skills. As a result, there is no agility and it is not user friendly for any non technical business users.
Fortunately, we have a data discovery tool which can easily complement our existing BI tool. Let's look and compare how a data discovery tool can impact on our data flow in an organization:
Data discovery tools have many advantages for the whole organization. It can be easy to connect with various data sources, and it doesn't require any technical skills from the business users. Users can combine and merge various data sources and visualize big data sets. They can easily find insights from data and use it for their decision making process.
In addition, let's now look at the benefits of using a data discovery tool for various roles in the organization:
Business management: It provides better information for the decision process, plus greater accuracy, richer detail, an enhanced ability to identify factors affecting business outcomes, and the connections among them.
IT managers: It eliminates the stream of individual support requests for analytics. It will free IT staff for other responsibilities and improves internal customer satisfaction.
IT staff: It allows them to dispense with the oodles of requests for data extracts.
Distinguished analysts: It gives greater productivity, fewer errors, and more time to spend on the more interesting aspects of data analysis. It allows better and easier data access, better integration among data sources, and the opportunity to probe data in greater detail. It provides a roader range of analytical methods.
Other data analysis roles: It provides access to a greater variety and depth of data, and to analysis methods.
Everybody else: It provides better information to support daily decision making at all levels of the organization.
We can continue this list with other roles in your organization and try to determine, what benefits they will get.
In my work, I use data discovery quite often, and I want to highlight some useful best practices:
Agility and rapid cycle iteration: It gives you the opportunity to rapidly discover any set of data and very quickly try a new hypothesis. In any case where the new hypothesis fails, you can quickly start with another new one.
Begin with the end in mind: Even if you don't know what you should do to find something in a particular data set, you can start to explore the data and get business insight. But it is highly recommended that you try to understand the business process.
Take advantages of data insights: During data exploration you may find valuable new information, for example, if you recognize a burst of high-volume sales for a new product. Immediately you can dive deep, and, for example, look to answer the question of, who is buying this product. We can dive deeper and try to understand more and more. Finally, we can create a new marketing segment based on your valuable insights.
Starting from this point, we will focus on learning SAP Lumira. First of all, we need to understand what SAP Lumira is exactly.
SAP Lumira is a family of data discovery tools which give us an opportunity to create amazing visualizations or even tell fantastic stories based on our big or small data. We can connect most of the popular data sources, such as Relational Database Management Systems (RDBMSs), flat files, excel spreadsheets or SAP applications. We are able to create datasets with measures, dimensions, hierarchies, or variables. In addition, Lumira allows us to prepare, edit, and clean our data before it is processed.
SAP Lumira offers us a huge arsenal of graphical charts and tables to visualize our data. In addition, we can create data stories or even infographics based on our data by grouping charts, single cells, or tables together on boards to create presentation- style dashboards. Moreover, we can add images or text in order to add details.
The following are the three main products in the Lumira family offered by SAP:
SAP Lumira Desktop
SAP Lumira Server
SAP Lumira Cloud
Lumira Desktop can be either a personal edition or a standard edition. Both of them give you the opportunity to analyse data on your local machine. You can even share your visualizations or insights via PDF or XLS.
Lumira Server is also in two variations—Edge and Server. As you know, SAP BusinessObjects also has two types of license for the same software, Edge and Enterprise, and they differ only in terms of the number of users and the type of license. The Edge version is smaller; for example, it can cover the needs of a team or even the whole department.
Lumira Cloud is Software as a Service (SaaS). It helps to quickly visualize large volumes of data without having to sacrifice performance or security. It is especially designed to speed time to insight. In addition, it saves time and money with flexible licensing options.
Here is a table which will help you to compare the various versions of SAP Lumira and choose the best one for your purposes:
|
Desktop |
Server |
Cloud | |||
|---|---|---|---|---|---|
|
Personal |
Standard |
Edge |
Enterprise |
Cloud | |
|
Access XLSX and CSV files |
+ |
+ |
+ |
+ | |
|
Access databases |
+ |
+ |
+ | ||
|
Access SAP HANA and BI universes |
+ |
+ |
+ | ||
|
Combine and transform data |
+ |
+ |
+ |
+ | |
|
Create visualizations, boards, and infographics |
+ |
+ |
+ |
+ |
+ |
|
Secure sharing |
+ |
+ |
+ | ||
|
Web / mobile support |
+ |
+ |
+ | ||
|
Free edition/trial |
+ |
+ |
+ |
+ |
+ |
Since we now have an idea what SAP Lumira is and how it can help us with data discovery or big data analytics, lets download and install SAP Lumira. In addition, we will also learn a bit about Lumira's nice, friendly interface. In this book, we will use the SAP Lumira Standard edition; you can download it with a trial key for a 60 days trial.
There are three methods for getting SAP Lumira:
Download it from http://www.saplumira.com
Download it from http://support.sap.com/swdc
Download from https://www.packtpub.com/support
This is the best choice because most of the books were written based on SAP Lumira Desktop version 1.21. It is possible to use the last version of SAP Lumira, which is very similar to its previous versions, but it can have some changes in design.
Before downloading and installing lets have a look at the system requirements of SAP Lumira:
|
Operating System |
32-bit |
64-bit |
|---|---|---|
|
Windows 7 SP1 |
+ |
+ |
|
Windows Server 2008 R2SP1 |
N/A |
+ |
|
Windows 8 / Windows 8.1 |
Not supported |
+ |
|
Windows Server 2012 / Windows Server 2012 R2 |
N/A |
+ |
The following table explains the hardware requirements:
|
Hardware requirements |
32-bit |
64-bit |
|
RAM (minimum) |
2 GB |
3 GB |
|
RAM (recommended) |
4 GB | |
|
Screen resolution (minimum) |
1280 x 720 | |
|
Free hard disk space needed for installation |
3.7 GB | |
|
Open local ports |
Local port 6401 should be open and you need at least one open local port in the port range: 4520-4539 | |
If your machine complies with these requirements, you can download it from http://saplumira.com/download/. In my case, I chose Free Download (64-bit).
You have just completed the first step in the amazing world of data discovery. You know a little bit about why we need data discovery, and you just downloaded the software which gives you outstanding capabilities to explore data and become a data geek.
First of all, we need to install SAP Lumira. There are two ways to accomplish this task:
Installing SAP Lumira with the wizard
Silent installation
I prefer using the wizard, but you can also go ahead with a silent installation, if you wish. We will see how to do it later in this chapter.
In order to run the wizard:
Find distributive and double click on SAPLumirainstall.exe.
Enter a destination folder, for example
C:\Program Files\SAP Lumira\,and click Next.Accept the license agreement and click Next.
Click Next and wait till the process finishes.
Click Finish and mark Launch SAP Lumira.
You just successfully installed SAP Lumira and launched it. It is ready to work. But first, let's take a tour around the interface.
In a silent installation the SAP Lumira installation wizard runs without displaying a user interface or prompting for user input; it reads the required input from a text file.
A silent installation is typically used by system administrators to push multiple installations across an infrastructure. After creating a silent installation response file, you can add the silent-installation command to your installation script.
If you want to perform a silent installation, you should create a response file, which consists of two parameters:
<InstallDir>—Then location where the SAP Lumira program files will be installed (required).<userkeycode>—License Key (optional).
In the text pad, just type:
InstallDir=C:\Program Files\SAP Lumira\
userkeycode=XXXXX-XX00000-XXXXXXX-XXXXXXX-XX2
Now we can start the silent installation:
Click Start->Run and enter
cmdto open the Run window.Navigate to, and double-click, the
SAPLumiraSetup.exeself-extracting archive file.For example, the file might be located at
C:\Program Files\SAP Lumira\SAPLumiraSetup.exe. EnterSAPLumiraSetup.exe -s -r <ResponseFilePath>\response.ini, where<ResponseFilePath>is the location where the response file will be saved. The parameter-rrequires the file name and location of the response file you created. The optional parameter-shides the self-extraction progress bar during a silent installation.
We have successfully installed SAP Lumira. Now we are going to get to know the user interface for Lumira. Go to Start->All Programs->SAP Business Intelligence->SAP Lumira.
SAP Lumira is a desktop application designed for acquiring, visualizing, and sharing data. The Home page of the application consists of four main areas, which have been marked in the following screenshot:
Let us take a look into each one of the 4 areas mentioned in the image:
Menu Bar (1): As with most desktop applications, there's a menu bar at the top of the screen that is available on every page, which gives us access to the core features of the application, grouped in File, Edit, View, Data, and Help menus.
Document (2): The document browser on the side enables quick access to the different types of items for which we can create a link to the SAP Lumira Cloud, and a list of the connections between documents and data sources. In addition, there is a My Items tab here, which use displays a list of all Lumira documents, Visualizations, Datasets, and Stories, which icons, labels, and subgroups to help distinguish between the different types.
Homepage (3): The home page displays a graphic that outlines the main workflow within SAP Lumira. The Acquire Data icon is actually a button that provides a quick way to navigate to the screens for acquiring data and creating a new document.
Bottom part of the homepage (4): At the bottom of the Home page, there are also links to samples, starter packs, communities, learning materials, and contacts.
Before you start working with data, it is good to set the preferences. Lets go to File->Preferences:
Using the file menu, we can open the Preferences dialog box to change the global settings for:
Viewing the data
Charting
Enriching data sets
Automating software updates
Defining network connections
Installing SQL drivers
Enabling geo map services through Esri ArcGis Online (Esri needs a separate license)
Using these options, we can customize the SAP Lumira interface. If you are not sure what to change, you can just try to experiment with the settings.
When we successfully finish installation of SAP Lumira, it will ask us to register the product. You can decline registration, but it will remind you each time, when you start the application.
You have the following registration options:
Enter a key code
Create a new SAP Lumira Cloud account
Use an existing SAP Lumira Cloud account
We can also can use the personal edition, but you will be restricted to certain data source types only.
SAP Lumira, as with other SAP products, is an enterprise solution. Despite the fact that it offers us a free personal edition of the software, this is not enough for all the capabilities which we want to learn. For example, we want to connect SAP BusinessObjects and SAP HANA, and we want to access databases because in real life most data is based in databases or in SAP applications.
We have several ways to get all capabilities of SAP Lumira:
Use the 30 days trial version
Buy a key
Download a bundle of trial keys from http://service.sap.com
The first two points are self explanatory, but I would like to elaborate a bit on the third. Every quarter, SAP offers their customers temporary keys for most of the SAP Products, such as SAP BusinessObjects, SAP Data Services, and so on. It is funny, but many enterprise companies run their business based on these trial keys. It is forbidden to do it, so be careful because SAP can visit you with an audit at any time. But for home use it is fine.
In order to download temporary keys, as well as other SAP Software, you need S-user. This is an account for the SAP portal. You can ask for access to the SAP portal through your SAP manager, if your company runs SAP, or you can try asking the guys on the SAP forum.
Tip
SAP S User ID is a unique number that SAP uses for identification of people on its web portals and databases. For example, you need an S User ID in order to access the SAP support portal or register for SAP certification. In this way, the S User ID is very similar to forms of authentication used by other large IT companies.
If you are a happy guy who has an S-user ID, I will show you how to download your bundle of trial keys:
Go to http://service.sap.com/support and log in to the portal using your S-user ID and password.
On the Home Page you will find the menu:
Help & Support: Here, you can search SAP notes about various errors or software issues. In addition, you can create a new ticket for the SAP support team; this is only available, if your company bought SAP support.
Software Downloads: This allows you you to search for SAP software and download it. In addition, you can find updates and service packs.
Keys & Requests: This is our menu, where we can find temporary keys.
Release & Upgrade Info: This is where you can find useful information about new versions of products.
All other menus are not that useful and we can miss them.
Click on Keys & Requests.
In the new window find Temporary key and click on it.
It will open a new window, and ask you to log in again. Keep calm, this is SAP.
Finally, it gives you the list of keys in PDF format.
Open the file and find the Lumira key in Section 1. Copy it.
Open the SAP Lumira application and go to Help->Enter Code. Paste the code.
We have activated SAP Lumira, and we are ready to explore the data and learn SAP Lumira. But, before we start, we should meet with one fake ecommerce retailer.
I have decided to demonstrate all of the capabilities of SAP Lumira through real-world cases. For this purpose, I came up with an e-commerce retailer—Unicorn Fashion.
This is an extremely fast growing ecommerce retailer, operating in Manitoba, Canada. It has headquarters in Winnipeg. There are more than 100 employees who work at Unicorn, in various departments. Unicorn is a good example of a data driven company. Every decision is based on data analysis. It is one of the key factors of its success. There are lots of departments that use corporate data for analysis:
Human resources: They analyse data for the employers, measure KPIs, and calculate bonuses and salaries
Top management: They use daily dashboards and analyse sales trends. Their business decisions, based on data
Inventory: They analyse prices, sizes, and brand. In addition, they calculate discounts for unpopular goods
Purchasing department: Their category managers analyse how to get their brands to perform and try to plan capacity for the next season
Marketing department: They plan, analyse, and measure marketing campaigns, and calculate marketing costs
Logistic department: They analyse the performance of various logistic companies, to try to reduce costs, and calculate the bonus for the couriers
Finance department: They calculate financial waterfall and main financial KPIs
Warehouse: They measure KPIs and plan capacity
In terms of software, Unicorn mostly uses SAP technologies. For example, their main BI tool is SAP BusinessObjects. And they use various data sources as a DWH, such as Teradata and SAP HANA. In addition, Unicorn analysts like to connect OLTP systems and grab some transactional data in order to make quick decisions.
Through out this book, we will help unicorn staff to get the maximum value from their data. For this purpose. we will learn about SAP Lumira and try to figure out how it can be used in your organization or just in daily work.
In this chapter we have looked at data discovery. You have learnt what data discovery is and why is it useful. In addition, we compared data discovery with traditional business intelligence in order to highlight the pros and cons of each technology. Moreover, we have successfully installed SAP Lumira on our machine and discussed various versions of SAP Lumira and types of registration. In addition, we have touched of the SAP Lumira interface and learnt about the Home menu. Finally, we met the Unicorn Fashion company. In the next chapter, we will start working with data more closely. Through exercises, we will connect to various data sources such as flat files, databases, and so on.
