Pentaho Analytics for MongoDB

Modern businesses capture huge volumes and varieties of data using several different data storage methods. There is no one-size-fits-all data storage method, because each technology has evolved to tackle the data challenges or opportunities of that specific time in history. We continue to see new and innovative data storage solutions as data volume, variety, and velocity grows, and as people figure out new ways to use data. The following is a small sampling of the variety of data sources you might encounter in a single organization:

Organizations invest heavily in these storage technologies and the skills needed to capture, store, and process data. MongoDB has emerged as a leader in the NoSQL category of databases. Because you are reading this book, you are probably well aware of the differences between MongoDB and relational databases; however, it is important to review and remind ourselves where MongoDB came from and why it is popular alternative to relational databases.

MongoDB is a document-oriented database designed to conquer some of the modern data storage challenges that developers and IT departments experience when using traditional relational databases. These modern data storage challenges started with the rise of the internet and high traffic websites such as Amazon and Google. These companies' websites attracted millions of users and subsequently massive volumes of website log, clickstream, and event data. Traditional relational database methods for handling growing data mostly involved scaling up (that is, vertical scaling) by adding more CPU and RAM to a single, often proprietary database server. This method of scaling was expensive and had limits on how far you could scale a single server. As a result, Google and Amazon decided to solve these data challenges by developing their own distributed data stores that could easily scale out (that is, horizontal scaling) across hundreds or thousands of commodity servers, as shown in the following figure. Horizontal scaling made it easier to scale dynamically by adding more machines to the cluster without any downtime or limits to compute capacity.

These early pioneers of distributed databases inspired a NoSQL data storage movement that included MongoDB. The term NoSQL is a popular way to describe MongoDB, because MongoDB does not use SQL, but NoSQL is not just about the query language. It has more to do with the way data is stored than just the query language. For many, the name NoSQL is inadequate, because it simply describes a query language and not the true essence of MongoDB, which is a horizontally scalable, distributed document database. Surprisingly, the name NoSQL originated simply as a way to describe these emerging distributed data stores using a short and unique Twitter hashtag, #nosql, for the purpose of advertising a meet up on the topic. As the story continued, the hashtag name stuck and is widely in use today!

Why would a database solution not leverage SQL, the most popular database query language used by developers and organizations all over the world? One key reason is that the SQL query language is not designed to efficiently query the nested constructs involved in hierarchical JSON documents, which form the foundation of the MongoDB document data model. JSON documents are language independent text files that represent data and are built on two primary data structures: nested collections of name/value pairs and ordered lists such as arrays. The following example shows the JSON representation of a dataset that describes the movie Forrest Gump. The JSON representation contains a parent object for movie information, a nested object representing the production company, and an array of cast member objects, shown as follows:

{
    "movie": "Forrest Gump",
    "rating": "PG-13",
    "duration_min": 142,
    "production_company": {
       "name": "Parmount Pictures",
       "streetAddress": "5555 Melrose Ave",
       "city": "Los Angeles",
       "state": "CA",
       "postalCode": 90038
    },
    "cast": [
        {
            "character": "Forrest Gump",
            "person": "Tom Hanks"
        },
        {
            "character": "Jenny Curran",
            "person": "Robin Wright"
        }
    ]
}

Each object is a comma-separated collection of key-value pairs enclosed in curly braces. MongoDB has its own query language built from the ground as a powerful way to retrieve, process, and update JSON documents. Document-oriented data storage is an alternative to SQL-based relational databases, and it offers some unique advantages that we will discuss in more detail in the next chapter. You can also learn more about JSON at json.org or Wikipedia.org/wiki/JSON.

MongoDB's use of JSON pairs each key with a complex data structure known as a document, and these documents can contain many different key-value pairs, key-array pairs, or even nested documents. MongoDB document-oriented data models enable the following benefits over relational databases:

Pentaho software consists of a suite of analytics products called Pentaho Business Analytics, providing a complete analytics software platform. This end-to-end solution includes data integration, metadata, reporting, OLAP analysis, ad-hoc query, dashboards, and data mining capabilities. The platform is available in two offerings: a community edition (CE) and an enterprise edition (EE). We will focus on the enterprise edition for this book because Instaview is included only in this edition of Pentaho.

Throughout the book, you will see that we group and refer to the platform in two major categories, Pentaho Data Integration (PDI) and Pentaho Business Analytics (BA). Even though we refer to PDI and BA as separate server categories, Pentaho tightly couples the PDI server with the BA server into a single platform offering. This unique approach helps companies solve their data integration challenges with multiple, diverse data sources including Big Data sources and instantly gain insight into business analytics for a broad set of users.

PDI gives users a graphical user interface to a parallel processing ETL engine to solve data integration challenges. The user interface reduces data integration complexity by eliminating the need to code data extractions, data transformations, and data loads. Some additional PDI benefits include:

BA provides web-based interfaces to create business models and interactive reports as well as analysis views and dashboards. The focus is on ease-of-use while providing a complete set of reporting and analysis capabilities that include the following web-based components:

It also includes the following client-based components:

As mentioned earlier, Pentaho is an early mover into the Big Data space as the first major BI vendor to extend its analytics platform with Big Data capabilities in May 2010. The partnership with MongoDB is one of the first few Big Data partnerships for Pentaho and since then, Pentaho continues to deliver MongoDB innovations. The Pentaho-MongoDB solution covers the entire Big Data life cycle from data extraction and preparation to data discovery, which we will explore throughout this book. Now that we have reviewed both technologies, it is time to install them on your computer.

Installing MongoDB on Windows or Linux is quick, easy, and well-documented in the MongoDB manual at http://docs.mongodb.org/manual/. We will install it on one Windows computer and restore the provided sample database for development purposes only. The database does not enable any authentication by default, so before you load any other personal or confidential data into MongoDB or consider it for your production environment, you will want to read about recommended security practices at http://docs.mongodb.org/manual/core/security.

The first thing we need to do is download the latest production release of the software. Included in the download are the MongoDB server, MongoDB shell, backup and restoration tools, import and export tools, and GridFS to manage large files that exceed the document size limit of 16 MB. MongoDB is self-contained without other system dependencies, so you can install and run MongoDB from any folder you want. Perform the following steps to set up and run the MongoDB server via the command prompt and set up MongoDB as a Windows service:

{
  "_id" :ObjectId("512d200e223e7f294d13d44c"),
  "id_session" : "71E1FF1A25B84045864FCEC447F8D012",
  "ip_address" : "47.54.245.196",
  "browser" : "Safari",
  "date_session" :ISODate("2013-01-01T17:36:32Z"),
  "duration_session" : 7.43,
  "referring_url" : "www.123.com"
}

{
  "_id" :ObjectId("512d1ff2223e7f294d13c0c4"),
"id_session" : "310FA578571D4D90847E33E8B894703D",
  "event" : "Visited Site"
}

Installing Pentaho on Windows, Linux, or Mac OS is quick and easy with the graphical installer option. This option, by default, installs the entire BA platform, PostGreSQLSolution database, Tomcat application server, and Sun JRE needed to provide a completely functioning analytics server. Typically, you would install the server components on a server and the client components on your workstation, but for development and training purposes, we are going to install all of the server and client components on a single Windows computer. The following list shows all of the platform components that will be installed on your computer:

Perform the following steps to download and install Pentaho on your Windows computer:

Congratulations! You should have a working installation of Pentaho, which installs the BA and DI servers as Windows systems services that will start automatically the next time you boot your computer. Please refer to the official Pentaho Installation Guide at http://infocenter.pentaho.com for detailed installation information. In the next chapter, we will explore the MongoDB data model and establish a connection from Pentaho to the sample MongoDB database.