Preface

Using Hidden Markov Models (HMMs) is a technique for modeling Markov processes with unobserved states. They are a special case of Dynamic Bayesian Networks (DBNs) but have been found to perform well in a wide range of problems. One of the areas where HMMs are used a lot is speech recognition because HMMs are able to provide a very natural way to model speech data. This book starts by introducing the theoretical aspects of HMMs from the basics of probability theory, and then talks about the different applications of HMMs.

Who this book is for

A basic understanding of probability theory, linear algebra, and calculus will make reading this book a lot easier. For the code examples, basic familiarity with Python programming is expected.

What this book covers

Chapter 1, Introduction to Markov Process, starts with a discussion of basic probability theory, and then introduces Markov chains. The chapter also talks about the different types of Markov chain classifying based on continuous or discrete states and time intervals.

Chapter 2, Hidden Markov Models, builds on the concept of Markov processes and DBNs to introduce the concepts of the HMM.

Chapter 3, State Inference – Predicting the States, introduces algorithms that can be used to predict the states of a defined HMM. The chapter introduces the Forward algorithm, the backward algorithm, the forward-backward algorithm, and the Viterbi algorithm.

Chapter 4, Parameter Inference Using Maximum Likelihood, discusses the basics of maximum likelihood learning. The chapter then moves on to applying maximum likelihood learning in the case of HMMs and introduces the Viterbi learning algorithm and Baum-Welch algorithm.

Chapter 5, Parameter Inference Using Bayesian Approach, starts by introducing the basic concepts of Bayesian learning. The chapter then applies these concepts in the case of HMMs and talks about the different approximation methods used for learning using the Bayesian method.

Chapter 6, Time Series Predicting, discusses the application of HMMs in the case of time series data. The chapter takes the example of the variation of stock prices and tries to model it using an HMM.

Chapter 7, Natural Language Processing, discusses the application of HMMs in the field of speech recognition. The chapter discusses two main areas of application: part-of-speech tagging and speech recognition.

Chapter 8, 2D HMM for Image Processing, introduces the concept of 2D HMMs and discusses their application in the field of image processing.

Chapter 9, Markov Decision Process, introduces the basic concepts of reinforcement learning and then talks about Markov decision process and introduces the Bellman equation to solve them.

To get the most out of this book

You need to have Python 3.4 installed on your machine in order to work through the chapters smoothly.

Download the example code files

You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

1. Log in or register at www.packt.com.
2. Select the SUPPORT tab.
3. Click on Code Downloads & Errata.
4. Enter the name of the book in the Search box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

• WinRAR/7-Zip for Windows
• Zipeg/iZip/UnRarX for Mac
• 7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Hands-On-Markov-Models-with-Python. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/9781788625449_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Now, let's define the MFCTagger class."

A block of code is set as follows:

from hmmlearn.hmm import GaussianHMM
import numpy as np
import matplotlib.pyplot as plt

Any command-line input or output is written as follows:

pip install matplotlib datetime

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "The possible states of the outcomes are also known as the domain of the random variable."

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [e-mail].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [e-mail] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packt.com.