What do you get with Print?

Instant access to your digital copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Redeem a companion digital copy on all Print orders

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Python Business Intelligence Cookbook

Chapter 1. Getting Set Up to Gain Business Intelligence

In this chapter, we will cover the following recipes:

Installing Anaconda
Installing, configuring, and running MongoDB
Installing Rodeo
Starting Rodeo
Installing Robomongo
Using Robomongo to query MongoDB
Downloading the UK Road Safety Data dataset

Installing Anaconda

Throughout this book, we'll be using Python as the main tool for performing business intelligence tasks. This recipe shows you how to get a specific Python distribution—Anaconda, installed.

Getting ready

Regardless of which operating system you use, open a web browser and browse to the Anaconda download page at http://continuum.io/downloads.

The download page will automatically detect your operating system.

How to do it…

In this section, we have listed the steps to install Anaconda for all the major operating systems: Mac OS X, Windows, and Linux.

Mac OS X 10.10.4

Click on the I WANT PYTHON 3.4 link. We'll be using Python 3.4 throughout this book.
Next, click on the Mac OS X — 64-Bit Python 3.4 Graphical Installer button to download Anaconda.
Once the download completes, browse your computer to find the downloaded Anaconda, and double-click on the Anaconda installer file (a .pkg file) to begin the installation.
Walk through the installer steps to complete the installation. I recommend keeping the default settings.
To verify that Anaconda is installed correctly, open a terminal and type the following command:
```
python
```
If the installer was successful, you should see something like this:

Windows 8.1

Click on the I WANT PYTHON 3.4 link. We'll be using Python 3.4 throughout this book.
Next, click on the Windows 64-Bit Python 3.4 Graphical Installer button to download Anaconda.
Once the download completes, browse your computer to find the downloaded Anaconda, and double-click on the Anaconda3-2.3.0-Windows-x86_64.exe file to begin the installation.
Walk through the installer steps to complete the installation. I recommend keeping the default settings.
To verify that Anaconda has installed correctly, open a terminal, or open a command prompt in Windows. Now type the following command:
```
python
```
If the installation was successful, you should see something like this:

Linux Ubuntu server 14.04.2 LTS

Linux servers have no graphical user interface (GUI), so you'll first need to log into your server and get a command prompt. With that complete, do the following:

On the Anaconda downloads page, select Linux.
Choose the Python 3.4 link.
Right-click on the Linux X 64-Bit button, and copy the link.
At the command prompt on your server, use curl to download the file, pasting the following download link:
```
curl –O <LINK TO DOWNLOAD>
```
I've created a special shortcut on my website that is a bit easier to type at the command line: http://robertwdempsey.com/anaconda3-linux.
Once Anaconda downloads, use the following command to start the installer:
```
bash Anaconda3-2.3.0-Linux-x86_64.sh
```
Accept the license agreement to begin installation.
When asked if you would like Anaconda to prepend the Anaconda3 install location to the PATH variable, type yes.
- To have the PATH update take effect immediately after the installation completes, type the following command in the command line:
```
source ~/.bashrc
```
Once the installation is complete, verify the installation by typing python in the command line. If everything worked correctly, you should see something like this:

How it works…

Anaconda holds many advantages over downloading Python from http://www.python.org or using the Python distribution included with your computer, some of which are as follows:

Almost 90 percent of what you'll use on a day-to-day basis is already included. In fact, it contains over 330 of the most popular Python packages.
Using Anaconda on both the computer you use for development and the server where your solutions will be deployed helps ensure that you are using the same version of the Python packages that your applications require.
It's constantly updated; so, you will always be using the latest version of Python and the Python packages.
It works on all the major operating systems—Linux, Mac, and Windows.
It comes with tools to connect and integrate with Microsoft Excel.

At the time of writing this, the current version of Anaconda for Python 3 is 2.3.0.

Installing, configuring, and running MongoDB

In this section, you'll see how to install, configure, and run MongoDB on all the major operating systems—Mac OS X, Windows, and Linux.

Getting ready

Open a web browser and visit: https://www.mongodb.org/downloads.

How to do it…

Mac OS X

The following steps explain how to install, configure, and run MongoDB on Mac OS X:

On the download page, click on the Mac OS X tab, and select the version you want.
Click on the Download (TGZ) button to download MongoDB.
Unpack the downloaded file and copy to any directory that you like. I typically create an Applications folder in my home directory where I install apps like this.
For our purpose, we're going to set up a single instance of MongoDB. This means there is literally nothing to configure. To run MongoDB, open a command prompt and do the following:
- At the root of your computer, make a data directory:
```
sudo mkdir data
```
- Make your user the owner of the directory using the chown command:
```
chown your_user_name:proper_group data
```
- Go to the directory where you have MongoDB.
- Go to the MongoDB directory.
- Type the following command:
```
./mongod
```
You should see the following output from Mongo:

Windows

The following steps explain how to install, configure, and run MongoDB on Windows:

Click on the Windows tab, and select the version you want.
Click on the Download (MSI) button to download MongoDB.
Once downloaded, browse to the folder where Mongo was downloaded, and double-click on the installer file.
When asked which setup type you want, select Complete
Follow the instructions to complete the installation.
Create a data folder at C:\data\db. MongoDB needs this directory in order to run. This is where, by default, Mongo is going to store all its database files.
Next, at the command prompt, navigate to the directory where Mongo was installed and run Mongo:
```
cd C:\Program Files\MongoDB\Server\3.0\bin
Mongod.exe
```
If you get any security warnings, give Mongo full access.
You should see an output like the following screenshot from Mongo, letting you know it's working:

Linux

The easiest way to install MongoDB in Linux is by using apt. At the time of writing, there are apt packages for 64-bit long-term support Ubuntu releases, specifically 12.04 LTS and 14.04 LTS. Since the URL for the public key can change, please visit the Mongo Installation Tutorial to ensure that you have the most recent one: https://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/.

Install Mongo as follows:

Import the public key:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --  recv 7F0CEB10

Create a list file for MongoDB:

echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release -sc)"/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list

Update apt:
```
sudo apt-get update
```
Install the latest version of Mongo:
```
sudo apt-get install -y mongodb-org
```
Run Mongo with the following command:
```
sudo service mongod start
```
Verify that MongoDB is running by checking the contents of the log file at /var/log/mongodb/mongod.log for a line that looks like this: [initandlisten] waiting for connections on port 27017
You can stop MongoDB by using the following mongod command:
```
sudo service mongod stop
```
Restart MongoDB with this command:
```
sudo service mongod restart
```
Note
MongoDB log file location
MongoDB stores its data files in /var/lib/mongodb and its log files in /var/log/mongodb.

How it works…

MongoDB's document data model makes it easy for you to store data of any structure and to dynamically modify the schema. In layman's terms, MongoDB provides a vast amount of flexibility when it comes to storing your data. This comes in very handy when we import our data. Unlike with an SQL database, we won't have to create a table, set up a scheme, or create indexes—all of that will happen automatically when we import the data.

Downloading the UK Road Safety Data dataset

In this section, we're going to download and take a bird's eye view of the dataset we'll be using throughout this book—the UK Road Safety Data. In total, this dataset provides more than 15 million rows across three CSV files.

How to do it…

Visit the following URL: http://data.gov.uk/dataset/road-accidents-safety-data/resource/80b76aec-a0a1-4e14-8235-09cc6b92574a.
Click on the red Download button on the right side of the page. I suggest creating a data directory to hold the data files.
Unpack the provided zip files in the directory you created.
You should see the following four files included in the expanded directory:
- Accidents7904.csv
- Casualty7904.csv
- Road-Accident-Safety-Data-Guide-1979-2004.xls
- Vehicles7904.csv

How it works…

The CSV files contain the data that we are going to use in the recipes throughout this book. The Excel file is pure magic, though. It contains a reference for all the data, including a list of the fields in each dataset as well as the coding used.

Coding data is a very important preprocessing step. Most analysis tools that you will use expect to see numbers rather than labels such as city or road type. The reason for this is that computers don't understand context like we humans do. Is Paris a city or a person? It depends. Computers can't make that judgment call. To get around this, we assign numbers to each text value. That's been done with this dataset.

Why we are using this dataset

It is said that up to 90 percent of the time spent on most data projects is for preparing the data for analysis. Anecdotal evidence from this author and those I speak with holds this to be true. While you will learn a number of techniques for cleaning and standardizing data, also known as preprocessing in the data world, the UK Road Safety Data dataset is an analysis-ready dataset. In addition, it provides a large amount of data—millions of rows—for us to work with.

This dataset contains detailed road safety data about the circumstances of personal injury road accidents in GB from 1979, the types (including Make and Model) of vehicles involved and the consequential casualties.

Key benefits

*Want to minimize risk and optimize profits of your business? Learn to create efficient analytical reports with ease using this highly practical, easy-to-follow guide.

*Learn to apply Python for business intelligence tasks—preparing, exploring, analyzing, visualizing and reporting—in order to make more informed business decisions using data at hand

*Learn to explore and analyze business data, and build business intelligence dashboards with the help of various insightful recipes

Description

The amount of data produced by businesses and devices is going nowhere but up. In this scenario, the major advantage of Python is that it's a general-purpose language and gives you a lot of flexibility in data structures. Python is an excellent tool for more specialized analysis tasks, and is powered with related libraries to process data streams, to visualize datasets, and to carry out scientific calculations. Using Python for business intelligence (BI) can help you solve tricky problems in one go. Rather than spending day after day scouring Internet forums for “how-to” information, here you’ll find more than 60 recipes that take you through the entire process of creating actionable intelligence from your raw data, no matter what shape or form it’s in. Within the first 30 minutes of opening this book, you’ll learn how to use the latest in Python and NoSQL databases to glean insights from data just waiting to be exploited. We’ll begin with a quick-fire introduction to Python for BI and show you what problems Python solves. From there, we move on to working with a predefined data set to extract data as per business requirements, using the Pandas library and MongoDB as our storage engine. Next, we will analyze data and perform transformations for BI with Python. Through this, you will gather insightful data that will help you make informed decisions for your business. The final part of the book will show you the most important task of BI—visualizing data by building stunning dashboards using Matplotlib, PyTables, and iPython Notebook.

What you will learn

* Install Anaconda, MongoDB, and everything you need to get started with your data analysis

* Prepare data for analysis by querying cleaning and standardizing data

* Explore your data by creating a Pandas data frame from MongoDB

* Gain powerful insights, both statistical and predictive, to make informed business decisions

* Visualize your data by building dashboards and generating reports

* Create a complete data processing and business intelligence system

What do you get with Print?

Instant access to your digital copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Redeem a companion digital copy on all Print orders

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Frequently bought together

$48.99

Python Data Visualization Cookbook (Second Edition)

$48.99

$43.99

Total $ 141.97

Filter reviews by

All

Amazon verified reviews

CS Mar 22, 2016

This book for all Python programmers wanting to implement BI functionalities. Well, it covers only one database, which is MongoDB, but you can always start with something. This book has good content and lot of recipes that cover setting up your environment, importing data, cleanig up of data and then analyzing it. Good book to start with, if you are into Python and BI functionalities.

Amazon Verified review

Rook501 Feb 11, 2016

This is precisely what I was looking for. Python is great with data and generating valuable business intelligence and this book does a great job getting you started.

Utsav Narayan Singh Jan 13, 2016

This book provides a good coverage of all topics for anyone planning to develop business intelligence solutions using Python. The books provides many recipies that cover setting up your environment, importing data, cleanig up of data and then analysing it. A promninent feature of the book is that it is platform agnostic and is suitable for everyone whether he is using Windows, Linx or OSX. I thouroughly enjoyed discovering new recipies in this book and jave used it as a reference from time to time.Note: I was reviewer of this book.

Alex Meadows Feb 15, 2016

Python Business Intelligence Cookbook is a good introduction to the basics of Pandas and working with csv files and MongoDB. One of the books few shortcomings is the lack of recipes showing interaction with a traditional RDBMS. While it can be understood that due to the variety and intermediate complexity that accessing a SQL database directly is out of scope for the book, it would have been nice to provide some reference to start with. That said, the various data types and process for analyzing, organizing, and visualizing data is spot on. The author even dives into predictive analytics. If you are looking for a general overview then this is a decent place to start.To note: I was asked by the publisher to provide this review in return for a free book. This did not impact my review.

Ed P Feb 09, 2016

If you are Python programmer introducing BI functionality to your applications then this would be a good starting point. The content is reasonably good, but for many of the tasks covered, similar results could be achieved much quicker and easier with dedicated Business Intelligence tools for ETL and reporting. Also, the database used is MongoDB. There is no coverage of many popular NoSQL and RDBMS systems such as MySQL, PostgreSQL, SQL Server, Hadoop and Oracle that are likely to be used in the corporate world. But as a starting point some of these recipes could be adapted to use other systems as well. All in all, a decent effort, but somewhat limited in applicability

Python Business Intelligence Cookbook: Leverage the computational power of Python with more than 60 recipes that arm you with the required skills to make informed business decisions

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Mac OS X 10.10.4

Windows 8.1

Linux Ubuntu server 14.04.2 LTS

Mac OS X

Windows

Linux

Note

Mac OS X

Windows

Why we are using this dataset

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs

Create a Free Account To Continue Reading

Sign in to activate your 7-day free access