Building Collaborative Filters

In the previous chapter, we mathematically defined the collaborative filtering problem and gained an understanding of various data mining techniques that we assumed would be useful in solving this problem.

The time has finally come for us to put our skills to the test. In the first section, we will construct a well-defined framework that will allow us to build and test our collaborative filtering models effortlessly. This framework will consist of the data, the evaluation metric, and a corresponding function to compute that metric for a given model.

Technical requirements

You will be required to have Python installed on a system. Finally, to use the Git repository of this book, the user needs to install Git.

The code files of this chapter can be found on GitHub:
https://github.com/PacktPublishing/Hands-On-Recommendation-Systems-with-Python.

Check out the following video to see the code in action:

http://bit.ly/2mFmgRo.

The framework

Just like the knowledge-based and content-based recommenders, we will build our collaborative filtering models in the context of movies. Since collaborative filtering demands data on user behavior, we will be using a different dataset known as MovieLens.

The MovieLens dataset

The MovieLens dataset is made publicly available by GroupLens Research, a computer science lab at the University of Minnesota. It is one of the most popular benchmark datasets used to test the potency of various collaborative filtering models and is usually available in most recommender libraries and packages:

MovieLens gives us user ratings on a variety of movies and is available in various sizes. The full version consists of more than...

User-based collaborative filtering

In Chapter 1, Getting Started with Recommender Systems, we learned what user-based collaborative filters do: they find users similar to a particular user and then recommend products that those users have liked to the first user.

In this section, we will implement this idea in code. We will build filters of increasing complexity and gauge their performance using the framework we constructed in the previous section.

To aid us in this process, let's first build a ratings matrix (described in Chapters 1, Getting Started with Recommender Systems and Chapter 5, Getting Started with Data Mining Techniques) where each row represents a user and each column represents a movie. Therefore, the value in the i^th row and j^th column will denote the rating given by user i to movie j. As usual, pandas gives us a very useful function, called pivot_table, to...

Item-based collaborative filtering

Item-based collaborative filtering is essentially user-based collaborative filtering where the users now play the role that items played, and vice versa.

In item-based collaborative filtering, we compute the pairwise similarity of every item in the inventory. Then, given user_id and movie_id, we compute the weighted mean of the ratings given by the user to all the items they have rated. The basic idea behind this model is that a particular user is likely to rate two items that are similar to each other similarly.

Building an item-based collaborative filter is left as an exercise to the reader. The steps involved are exactly the same except now, as mentioned earlier, the movies and users have swapped places.

Model-based approaches

The collaborative filters we have built thus far are known as memory-based filters. This is because they only make use of similarity metrics to come up with their results. They learn any parameters from the data or assign classes/clusters to the data. In other words, they do not make use of machine learning algorithms.

In this section, we will take a look at some filters that do. We spent an entire chapter looking at various supervised and unsupervised learning techniques. The time has finally come to see them in action and test their potency.

Clustering

In our weighted mean-based filter, we took every user into consideration when trying to predict the final rating. In contrast, our demographic-based...

Summary

This brings us to the end of our discussion on collaborative filters. In this chapter, we built various kinds of user-based collaborative filters and, by extension, learned to build item-based collaborative filters as well.

We then shifted our focus to model-based approaches that rely on machine learning algorithms to churn out predictions. We were introduced to the surprise library and used it to implement a clustering model based on kNN. We then took a look at an approach to using supervised learning algorithms to predict the missing values in the ratings matrix. Finally, we gained a layman's understanding of the singular-value decomposition algorithm and implemented it using surprise.

All the recommenders we've built so far reside only inside our Jupyter Notebooks. In the next chapter, we will learn how to deploy our models to the web, where they can be used...