Episode 4: What’s Data Science got to do with Netflix’s recommender system?

Advances in data science have changed the way we communicate, share, and receive information. Consider how big data has changed our TV and movie experiences for instance. Companies like Netflix collect thousands of data points from several places to make suggestions to users with the help of a tool known as a recommender engine.

Netflix’s recommendation algorithm has a RMSE (root mean squared error) of 0.88, which basically means that the differences between suggestions presented by this model and the user needs are hardly any. This state-of-the-art algorithm is the result of a machine learning and data mining competition with a 1 million dollar prize money. But how did the winning engineering team build it? And how does it work?

Before we dive deeper, let’s say that Netflix is a subscription based streaming service and thus its success can be seen as a function of acquisition rate of new users, cancellations and the rate at which former members rejoin. Simply put, the more members Netflix has, the higher its revenue. That’s why the company holds dear the satisfaction of its customers and it has the recommendation algorithm to thank for it. To demonstrate how important the recommendation engine is for Netflix’s incomes, 80% of stream time is achieved through this system which translates to 1 Billion dollars of savings on customer acquisition.

Netflix uses machine learning to give the platform the ability to automate millions of decisions based on user activities. When Netflix recommends “How I Met Your Mother” because I like “Friends”, machine learning was behind that decision. This suggestion is the Netflix recommendation engine at work: it uses your past activity and returns movies and shows it thinks you would enjoy. But how is that any different from any other recommendation system?

There are multiple potential methods for creating a recommendation algorithm. A basic implementation would be the editorial method in which the platform would make suggestions based on a small number of users such as critics. Another easy one is named the simple collection method where suggestions are based on the top products across the platform. Netflix doesn’t use these methods because they do not allow personalization, nor cover the breadth of the movie catalogs and user preferences. Instead, Netflix slays the recommendation game by the use of personalized methods where movies and shows are suggested to the users who are most likely to enjoy them based on their personal ratings and people with similar interests. This way, the Netflix methodology accounts for the diversity in its audiences and its very large catalog.

Thanks to machine learning, the Netflix engineering team was able to create a “smart” platform that calculates the likelihood of a user liking a product. To understand the probability aspect of the algorithm, let’s consider this example: a utility matrix that is obtained by placing a score on the relationship between users and a movie type to predict their preferences.

Each user has seen titles on Netflix, liked some and disliked some. Based on the utility matrix, Netflix may recommend to user A a “whodunit” since they liked the thrilling plot of a horror movie they have seen recently whereas the algorithm would recommend a soft and light-hearted movie to user B because they did not like a horror movie they saw last week and give user C romance movies suggestions. And thus, each user will receive a suggestion that’s personalized for them!

Netflix interface also uses video ranking algorithms like Personalized Video Ranker (or PVR) which usually filters down the catalog by a certain criteria (violent, romance, thriller…), Top-N Video Ranker which is similar to PVR but looks only at the head of the ranking and Trending Now Ranker which captures temporal trends which Netflix deduces to be strong predictors. These rankers are used to form a row based interface where it is easy to get feedbacks as a right scroll on a row would indicate interest whilst a scroll down would indicate non-interest or irrelevance.

In a nutshell, Netflix’s recommender system collects a huge amount of data from customer behavior that is then used by machine learning algorithms in order to perfect the user experience.

Resources:

How does Netflix recommend movies? matrix factorization

Deep Dive into Netflix’s Recommender System