The modern market for Business Intelligence (BI) tools is full of various products with different advantages and disadvantages. For example, some of them are easy to deploy but hard to maintain. Another has strong data discovery capabilities but restricted data governance and struggles with complex enterprise data models. MicroStrategy 10 is one of the leading platforms on the market, can handle all data analytics demands, and offers a powerful solution. It combines self-service BI, data discovery and preparation, and big data exploration, bringing the vision of a single-vendor solution within reach of BI and analytics leaders. It combines data discovery and enterprise BI in a single platform, allowing organizations to invest in a single vendor that provides both agile self-service BI and traditional reporting, with integration between the two.
This chapter will cover:
BI best practices
BI maturity model
MicroStrategy 10 new features
Deploying MicroStrategy 10 on top of Linux
Deploying business case
According to a Gartner survey conducted among 2,800 CIOs of top companies, BI and analytics remain the number one investment priority. MicroStrategy has done a great job since 1991. MicroStrategy is a leader in Enterprise BI and can handle the modern demand for BI and analytics.
MicroStrategy 10 can easily handle modern hot topics such as big data analytics, predictive analytics, and mobile analytics. It is easily scalable and secure.
The main roles of BI are to discover new opportunities, improve processes, help to make the right decisions at the right time and reduce operational costs. As a result, BI tools should improve business performance and help to meet organizational goals, such as higher revenue, lower costs, and decreased risks.
Nowadays, we see lots of BI tools. Sometimes, we have to ask "What is the best BI tool?" In order to find the answer, we can look at reports from two main analytical agencies: Gartner and Forrester. Here is the Gartner Magic Quadrant for BI & Analytics Platforms for 2015:
This quadrant shows us the leaders in the market. But do not trust this rating, because it is a kind of marketing. The best way to understand the pros and cons of any BI tool is practically; just download and install a trial version and play with it. Most tools, especially the leading ones, have common functionality or similar features. A good example is the interface of Tableau. It is so good that other companies have tried to reproduce it in their own products.
Every organization uses BI. The question is, just how good are they? Even if the organization does not have BI at all, it at least uses spreadsheets to consolidate numbers. This is also a kind of BI. We can observe many levels of BI maturity. Let us try to demonstrate this, based on four phases shown here:
The first phase is Scattered Reporting, in other words, the organization doesn't have any BI solution and even doesn't try one. Various departments grab data from transaction systems and try to figure out what's going on. We see this situation especially in startups or small companies. It is the most inefficient way to measure business and make decisions.
If you are at the beginning of your analytics journey, then we highly recommend you meet Lean Analytics methodology.
The next phase is Centralized BI Solution. It is the most common approach nowadays, when organizations build corporate data warehouses, integrate data from internal sources and systems, deploy a BI solution, and try to force everyone to use a single point of truth. It is a long journey to leave this phase, and sometimes it is an endless process because business constantly changes.
The third phase is Big Data Analytics. Despite the fact that the term big data is almost gone, we still use it, because it is a very popular phrase that means lots of unstructured data such as social data, open source data, and so on. In other words, organizations try to enrich their internal data in order to find more insights. As a result, companies are forced to use new technologies such as Hadoop or NoSQL datastores. In addition, companies start to use machine learning technologies and try to make them part of their BI framework.
The last phase is Real-time Mobile Business Insight. This phase means that finally organizations can focus on the speed of delivery of business insights, their mobility, and predictive capabilities.
Of course, this maturity model is not standard, we just tried to structure BI maturity levels based on our experience. MicroStrategy can handle even the final phase, a real competitive advantage.
Every BI project has its own life cycle. Let us look at a common life cycle:
The first step is Analyze Business requirements, which means that we should gather the business requirements and transform them into functional and non-functional specifications, create a template for reports, and so on. The next step is Design the logical data model, where we build a logical data model based on business requirements, which shows the business entities and the relationships between them. The third step is Design the physical data model, where we transform the logical data model into a physical data model which defines the structure of the data warehouse. The fourth step is Build the data warehouse. In this step, we create the data warehouse, build data marts, and load data.
On the fifth step, Create the Project, we start to work directly in MicroStrategy, where we define schema, attributes, facts, hierarchies and so on. All this information is stored in metadata, the core of MicroStrategy, in a relation database. The sixth step is Develop Reports/Documents and of course dashboards. In addition, we share our insights across the organization using various channels such as email, FTP, and so on.
MicroStrategy is a market leader in BI products. It has rich functionality in order to meet the requirements of modern businesses. In 2015, MicroStrategy provided a new release of MicroStrategy, version 10. It offers both agility and governance like no other BI product. In addition, it is easy to use and enterprise ready. At the same time, it is great for both IT and business. In other words, MicroStrategy 10 offers an analytics platform that combines an easy and empowering user experience, together with enterprise-grade performance, management, and security capabilities. It is true bimodal BI and moves seamlessly between styles:
Data discovery and visualization
Enterprise reporting and dashboards
In-memory high performance BI
Scales from departments to enterprises
Administration and security
MicroStrategy 10 consists of three main products: MicroStrategy Desktop, MicroStrategy Mobile and MicroStrategy Web.
MicroStrategy Desktop lets users start discovering and visualizing data instantly. It is available for Mac and PC. It allows users to connect, prepare, discover, and visualize data. In addition, we can easily promote to a MicroStrategy Server. Moreover, MicroStrategy Desktop has a brand new HTML5 interface and includes all connection drivers. It allows us to use data blending, data preparation, and data enrichment. Finally, it has powerful advanced analytics and can be integrated with R.
To cut a long story short, we want to notice the main changes of the new BI platform. All developers keep the same functionality, the looks as well as architect the same. All changes are about web interface and Intelligence Server. Let's look closer at what MicroStrategy 10 can show us.
MicroStrategy 10 expands the analytical ecosystem by using third-party toolkits such as:
Data visualization libraries: We can easily plug in and use any visualization from the expanding range of Java libraries
Statistical toolkits: R, SAS, SPSS, KXEN, and others
Geolocation data visualization: This uses mapping capabilities to visualize and interact with location data
MicroStrategy 10 has more than 25 new data sources that we can connect to quickly and simply. In addition, it allows us to build reports on top of other BI tools, such as SAP Business Objects, Cognos, and Oracle BI. It has a new connector to Hadoop, which uses the native connector. Moreover, it allows us to blend multiple data sources in-memory.
We want to notice that MicroStrategy 10 has got reach functionality for work with data such as:
Streamlined workflows to parse and prepare data
Multi-table in-memory support from different sources
Automatically parse and prepare data with every refresh
100+ inbuilt functions to profile and clean data
Create custom groups on the fly without coding
In terms of connection to Hadoop, most BI products use Hive or Impala ODBC drivers in order to use SQL to get data from Hadoop. However, this method is bad in terms of performance. MicroStrategy 10 queries directly against Hadoop. As a result, it is up to 50 times faster than via ODBC.
Let's look at some of the main technical changes that have significantly improved MicroStrategy. The platform is now faster than ever before, because it doesn't have a two-billion-row limit on in-memory datasets and allows us to create analytical cubes up to 16 times bigger in size. It publishes cubes dramatically faster. Moreover, MicroStrategy 10 has higher data throughput and cubes can be loaded in parallel 4 times faster with multi-threaded parallel loading. In addition, the in-memory engine allows us to create cubes 80 times larger than before, and we can access data from cubes 50% faster, by using up to 8 parallel threads. Look at the following table, where we compare in-memory cube functionality in version 9 versus version 10:
Number of rows
Any schema, tabular or multiple sets
In order to make the administration of MicroStrategy more effective in the new version, MicroStrategy Operation Manager was released. It gives MicroStrategy administrators powerful development tools to monitor, automate, and control systems. Operations Manager gives us:
Centralized management in a web browser
Enterprise Manager console within Tool
Triggers and 24/7 alerts
System health monitors
Multiple environment administration
MicroStrategy 10 offers new training courses that can be conducted offline in a training center, or online at http://www.microstrategy.com/us/services/education. We believe that certification is a good thing on your journey. The following certifications now exist for version 10:
MicroStrategy 10 Certified Associated Analyst
MicroStrategy 10 Certified Application Designer
MicroStrategy 10 Certified Application Developer
MicroStrategy 10 Certified Administrator
After passing all of these exams, you will become a MicroStrategy 10 Application Engineer. More details can be found here: http://www.microstrategy.com/Strategy/media/downloads/training-events/MicroStrategy-certification-matrix_v10.pdf.
Let us briefly look at the history of MicroStrategy, which began in 1991:
1991: Released first BI product, which allowed users to create graphical views and analyses of information data
2000: Released MicroStrategy 7 with a web interface
2003: First to release a fully integrated reporting tool, combining list reports, BI-style dashboards, and interface analyses in a single module.
2005: Released MicroStrategy 8, including one-click actions and drag-and-drop dashboard creation
2009: Released MicroStrategy 9, delivering a seamless consolidated path from department to enterprise BI
2010: Unveiled new mobile BI capabilities for iPad and iPhone, and was featured on the iTunes Bestseller List
2011: Released MicroStrategy Cloud, the first SaaS offering from a major BI vendor
2012: Released Visual Data Discovery and groundbreaking new security platform, Usher
2013: Released expanded Analytics Platform and free Analytics Desktop client
2014: Announced availability of MicroStrategy Analytics via Amazon Web Services (AWS)
2015: MicroStrategy 10 was released, the first ever enterprise analytics solution for centralized and decentralized BI
We know only one way to master MicroStrategy, through practical exercises. Let's start by downloading and deploying MicroStrategy 10.2.
In order to master MicroStrategy and learn about some BI considerations, we need to download the all-important software, deploy it, and connect to a network. During the preparation of the training environment, we will cover the installation of MicroStrategy on a Linux operating system. This is very good practice, because many people work with Windows and are not familiar with Linux, so this chapter will provide additional knowledge of working with Linux, as well as installing MicroStrategy and a web server. Look at the training architecture:
There are three main components:
Red Hat Linux 6.4: Used for deploying the web server and Intelligence Server.
Windows machine: Uses MicroStrategy Client and Oracle database.
Virtual machine with Hadoop: Ready virtual machine with Hadoop, which will connect to MicroStrategy using a brand new connection.
In the real world, we should use separate machines for every component, and sometimes several machines in order to run one component. This is called clustering. Let's create a virtual machine.
Let's create a virtual machine with Red Hat Linux, which will host our Intelligence Server:
Go to http://www.redhat.com/ and create an account
Go to the software download center: https://access.redhat.com/downloads
Red Hat Enterprise Linux Server
Red Hat Enterprise Linux 6.4 x86_64
Now we can create a virtual machine with RHEL 6.4. We have several options in order to choose the software for deploying virtual machine. In our case, we will use a VMware workstation.
Before starting to deploy a new VM, we should adjust the default settings, such as increasing RAM and HDD, and adding one more network card in order to connect the external environment with the MicroStrategy client and sample database. In addition, we should create a new network.
When the deployment of the RHEL virtual machine is complete, we should activate a subscription in order to install the required packages. Let us do this with one command in the terminal:
# subscription-manager register --username <username> --password <password> --auto-attach
According to the installation and configuration guide, we should deploy all necessary packages. In order to install them, we should execute them under the root:
# su # yum install compat-libstdc++-33.i686 # yum install libXp.x86_64 # yum install elfutils-devel.x86_64 # yum install libstdc++-4.4.7-3.el6.i686 # yum install krb5-libs.i686 # yum install nss-pam-ldapd.i686 # yum install ksh.x86_64
Moreover, we should change the parameters of the kernel:
# nano/etc/sysctl.conf kernel.sem=250 32000 32 2048 kernel.shmmni = 4096 vm.max_map_count=5242880
Then, save and reboot.
In order to download MicroStrategy software, we should already have an account on the MicroStrategy Resource Portal at https://resource.microstrategy.com, or we can create one using a corporate email address. Even if we do not have a license for MicroStrategy, we can obtain one for 30 days. That is enough to finish this book and learn the new features of MicroStrategy 10.
MicroStrategy 10.2 is available, and we are going to use this version in the book.
Let's download distributive
10.2.0008.0052_10.2_GA_Linux.tar and put it on the virtual machine with Linux
Let's create a folder for MicroStrategy using the root user and change its rights in order to extract the archive into this folder:
# mkdir mstr10 # chmod 777 mstr10
Then we should extract the MicroStrategy distributive into our new folder. We can use the default Archive Manager.
After successful extraction, we should create four folders for MicroStrategy, in order to deploy the software and write the installation logs.
Using the root user, we can run the following commands:
# mkdir /mstr10/microstrategy # mkdir /mstr10/microstrategy_home # mkdir /mstr10/microstrategy_logs # mkdir /mstr10/microstrategy_health_agent
By default, there are three methods of installation for MicroStrategy Server:
GUI: The most common approach, using a graphical interface and navigation with a mouse
Command line: The approach for terminal-only machines
Silent installation: Automatic installation process that does not need human participation, but it is important to think about parameters and settings
In Red Hat, we have a GUI and can use its benefits. Let's install:
# cd /mstr10/Installations/QueryReportingAnalysis_Linux # ./setup.sh user name Microstrategy name masteringbi
Copy and paste your license key.
Choose the destination locations that we have already created:
All the other parameters we can leave as is.
After successful installation of MicroStrategy 10.2, we should set up a MicroStrategy repository and connect it to the Intelligence Server. In our case, we will use a PostgreSQL database on the RHEL virtual machine for the repository.
Let's download and install the PostgreSQL database server:
# yum install postgresql-server # service postgresql initdb # chkconfig postgresql on # service postgresql start # s # psql
Let's create a new database in PostgreSQL for MicroStrategy metadata. In order to do this, we should go to the terminal and run the following:
# psql # CREATE DATABASE mstrmd; # CREATE USER mstr PASSWORD 'happy2016'; # GRANT ALL ON DATABASE mstrmd TO mstr;
In addition, we should register the plpgsql language in our new database:
# \c mstrmd # CREATE TRUSTED LANGUAGE plpgsql;
By default, PostgreSQL uses IDENT-based authentication, and this will never allow you to log in with a username and password. We should change the method of authentication by modifying the
pg_hba.conf file. Go to this folder:
# cd /var/lib/pgsql/data # nano pg_hba.conf
Replace IDENT on trust and restart PostgreSQL:
# service postgresql restart
Using the MicroStrategy connectivity wizard, we can create a new DSN. In order to open the wizard, we should run the following commands:
# cd /mstr10/microstrategy_home/bin # ./mstrconnectwiz
Let's choose PostgreSQL ODBC driver and fill in the form with our credentials:
MicroStrategy has a configuration file that keeps all DSNs. We can create a new DSN directly in this file. We can look at our new DSN like this:
# vi /mstr10/microstrategy_home/odbc.ini
Here is our DSN metadata. In addition, we can configure the parameters of our connection in order to increase performance.
In order to create metadata tables in the brand new database, we should run Configuration Wizard:
# cd /mstr10/microstrategy_home/bin # ./mstrcfgwiz
During this process, wizard will automatically:
Create metadata tables and history list tables
Configure Intelligence Server
MicroStrategy metadata is a core piece of software. In other words, the number of tables in a database that contain a definition of all MicroStrategy objects.
In Configuration Wizard, we should create a new database and choose our DSN:
The next step is the configuration of Intelligence Server using fresh metadata tables. We can use
masteringbi as a server definition. After successful configuration we can close wizard and deploy MicroStrategy Web.
We need a web server in order to deploy MicroStrategy Web and other components, such as Mobile Server and Operations Manager. For this book, we have chosen the Tomcat web server as our web application server.
Let us install Tomcat on our RHEL server. Open the terminal and give the following commands:
Install Java if it is needed:
# yum install java-1.8.0-openjdk # java -version openjdk version "1.8.0_65"
Download and deploy the Tomcat server:
# cd /tmp # wget http://www.us.apache.org/dist/tomcat/tomcat-7/v7.0.67/ bin/apache-tomcat-7.0.67.tar.gz # tar xzf apache-tomcat-7.0.67.tar.gz # mv apache-tomcat-7.0.67 /usr/local/tomcat7
# cd /usr/local/tomcat7 # ./bin/startup.sh
Create a Tomcat user for MicroStrategy Web:
# cd /usr/local/tomcat7/conf # nano tomcat-users.xml <role rolename="admin"/> <role rolename="manager-gui"/> <user username="admin" password="admin" roles="admin,manager-gui"/>
Then, restart Tomcat.
In order to access Tomcat, we should go to
Now we can deploy MicroStrategy Web and Operations Manager on a brand new Tomcat server. Let's do it:
# cp /mstr10/microstrategy/OperationsManager/MicroStrategyWeb.war /usr/local/tomcat7/webapps # cp /mstr10/microstrategy/OperationsManager/MicroStrategyOM.war /usr/local/tomcat7/webapps
Then we need restart Tomcat:
# /usr/local/tomcat7/bin/shutdown.sh # /usr/local/tomcat7/bin/startup.sh
After deploying, we should go to the web admin:
Then, connect to our server. We can do this just by adding the localhost server. In properties, we should mark that it automatically connects to Intelligence Server.
Later in the book, we will create projects and they will be available here:
In addition, we can go to Operations Manager in order to configure it:
If you change the war filename, you will have to use a different URL. For example,
mstr.war the URL will be
The Operation Manager will be configured in Chapter 9, MicroStrategy System Administration.
We have finished the installation and configuration of Intelligence Server. Now we need to install the MicroStrategy Client tools on the Windows machine.
We should open a port in the firewall for Tomcat on the Linux machine, in order for the Windows machine to access it:
# nano /etc/sysconfig/iptables
Add a new line for port
-A INPUT -m state --state NEW -m tcp -p tcp --dport 8080 -j ACCEPT
We should also open a port for Intelligence Server:
-A INPUT -m state --state NEW -m tcp -p tcp --dport 34952 -j ACCEPT
Then, restart the service:
# service iptables restart
As a result, we can access MicroStrategy from Windows.
According to our training architecture, we will use at least two machines. One is a Windows machine for client tools and the Oracle database, and a Linux machine for Intelligence Server and the PostgreSQL repository database. Let's look at how we can quickly create a network. This knowledge is very important for BI guys, because usually BI software has complex architecture and communicates with each other through the network. In our small example, the reader can learn basic considerations for building a network between several machines. Let's build a network using VMware:
At the top of the VMware Workstation desktop, select Edit and then select Virtual Network Editor.
Change the settings on this screen to match these:
Enable Use local DHCP service to distribute IP address to VMs.
Click the OK button. Now the preset IP address included in the images should work for VMware Workstation and VMware Player.
Open Control Panel and locate the Network and Sharing Center option. Double-click on it.
The private network is the bridged adapter and the public network is host-only. If your image is not making connections, look at the properties of the public network. Select the Local Area Connection 2 link:
Click the Properties button.
Select the Internet Protocol Version 4 (TCP/IPv4) entry and then click the Properties button.
Now we should change the network for Red Hat also. Let's do it:
Go to the terminal and run the
system-config-networkutility to change the IP address:
With the red highlight on the Device Configuration option, press the Enter key.
If you need to update the IP address, use the cursor keys to move up and down through the fields and then you can make the update. Use the cursor keys or the Tab key to move to the Ok button and press the Enter key or the spacebar to select Ok.
After you work your way back to the first screen, you'll have to use the Tab key to navigate to the Save & Quit button:
Restart the network service:
# service network restart
Check the result:
As a result, the Linux and Windows machines now live on one network and can easy communicate with each other. Using this approach, we can connect other machines that run other software. For example, we could run Oracle on a separate machine. In addition, we can create a cluster of several machines running MicroStrategy.
MicroStrategy prefers up to four nodes in a cluster, but you can install more than four if you want.
Now we can install the MicroStrategy Client tools and connect Intelligence Server.
One of the simplest tasks is installing the client tools on the Windows machine. We should install the following software:
MicroStrategy Server Administrator
MicroStrategy Object Manager
MicroStrategy Command Manager
MicroStrategy System Manager
MicroStrategy Integrity Manager
We should connect to our Intelligence Server, which is based on RHEL. Let's run Developer on port
34952 with the server name as the IP address
192.168.81.134. There are not any projects yet, but soon we will create one good project using our great expertise.
For our book, we decided to use Oracle 11g because it is the most popular RDBMS in the world. We can download Oracle from http://www.oracle.com/technetwork/database/enterprise-edition/downloads/112010-win64soft-094461.html.
In our case, we will install it on the same machine as MicroStrategy, in order to save resources. In the real world, we should use a separate server. The installation process is pretty simple and straightforward. Just download the software and run
The last step before finishing this chapter is to deploy our sample database, which we will use throughout this book in order to build advanced visualizations, dashboards, and many other things.
There are SQL files in the attachment that comes with this book. We are going to deploy these in our Oracle database:
In order to deploy these files in the Oracle database, we should:
Connect to the Oracle database as a system user.
Run these commands:
In the next chapter, we will create a schema based on this database.
We want to share some of our tips that help to improve perception of BI users and make their lives a little bit easier. We also want to share some great books and training that can help to increase skills and expertise, in order to grow within your organization or find a better job.
We want to share some important skills from experienced BI developers. These cover several areas:
BI software: There are many BI tools on the market but they have many features in common. That's why it is good to master MicroStrategy, then it is easy to understand how others work.
Dimensional modeling: This approach is the most popular design schema.
Data modeling: Before starting to develop schema, we should understand the corporate data model, that's why we should learn how to read models.
Databases: Nowadays, relational databases are the most popular choice for data warehouses. As BI tools, there are plenty of databases, but all of them run SQL and have different advantages and disadvantages.
Network knowledge: Usually BI tools use three-tier architecture; as a result it is good to know how to set up the network, open the network ports, and so on.
Visualization: It is a well-known fact that graphical representation of data is the most effective way to represent data.
Security: There are many users who work with BI and have different levels of access. It is very important to understand security topics and different ways of authenticating users.
Unix operational systems: Usually big companies prefer to use Unix systems as their main operating system for software, because they are reliable. However, they have a console or a terminal, which is why it is good to have knowledge of working with such operating systems.
There is no secret to how to master BI. There is only one way - constantly learn through reading and technical exercises. www.packtpub.com offers us plenty of technical books that can help us to solve any technical issues.
The most critical skill for a BI developer is to quickly find a solution using Google. It is impossible to store all relevant information in our head, but we can easily Google it and find the right solution. Do not hesitate to ask Google if you are struggling.
One hot topic is the happiness of business users. Usually they are not technical guys and often struggle with report development. Another difficulty is that there are many attributes and metrics in the data warehouse, and sometimes we can't use them in one query because it could produce cross joins and even crash the database. There is only one solution - constantly train them and give them the opportunity to learn the technical and business sides of BI solutions. As a result, the BI department should develop two streams of training - business and technical. Technical training relates to BI software. For example, our book is an advanced training for MicroStrategy. At the same time, business users should learn different business cases. For example, how they can build reports in order to compare revenue for 2014 and 2015 across all branches in Canada?
The main challenge of this training is to motivate business users. But there is a lack of time to learn. As a result, they make wrong decisions on inaccurate data. In order to involve them in the training process, we want to share one of our success stories.
One of the authors implemented BI Academy. It was an internal brand; users who successfully finish the training and pass the internal exam get a branded T-shirt:
In order to make the training process fun, the author used special software that offers the opportunity to create cartoons:
These guys were very popular because they helped to solve real business problems. Another piece of advice is to contact heads of department and encourage them to make this online training mandatory for employees. There are some training courses based on this approach on YouTube: https://www.youtube.com/channel/UCAGzL89m1L3c675hn7DE2Uw
Or you can see the MicroStrategy example here:
The final thing that is good to have in an organization is a BI portal. When companies run BI solutions, they do it for users. Usually, BI users are not technical. The best practice is to build BI and integrate it into an intranet website. For example, in the attachment to this chapter, there is a file called
sweet home.html. It is not a BI portal, but it demonstrates the idea very well. It helps business users and developers to quickly get the latest news about BI changes and innovations. Moreover, it links to BI environments and training. For example, the company can use Microsoft SharePoint because it is easy to deploy, and we can easily integrate it with MicroStrategy and Office products.
In this chapter, we talked about BI and shared our thoughts on some BI-related topics, as well as some tips that can improve the BI ecosystem. We then met MicroStrategy 10 and its new features. Finally, we deployed MicroStrategy 10 on top of Red Hat Linux and Tomcat. In addition, we deployed Oracle and a sample database.
In the next chapter, we are going to create a semantic layer and learn best practices for Oracle integration, as well as the creation of schema and schema objects.