Advances in data science have changed the way we communicate, share, and receive information. Consider how big data has changed our TV and movie experiences for instance. Companies like Netflix collect thousands of data points from several places to make suggestions to users with the help of a tool known as a recommender engine.
Netflix’s recommendation algorithm has a RMSE (root mean squared error) of 0.88, which basically means that the differences between suggestions presented by this model and the user needs are hardly any. This state-of-the-art algorithm is the result of a machine learning and data mining competition with a 1 million dollar prize money. But how did the winning engineering team build it? And how does it work?
Before we dive deeper, let’s say that Netflix is a subscription based streaming service and thus its success can be seen as a function of acquisition rate of new users, cancellations and the rate at which former members rejoin. Simply put, the more members Netflix has, the higher its revenue. That’s why the company holds dear the satisfaction of its customers and it has the recommendation algorithm to thank for it. To demonstrate how important the recommendation engine is for Netflix’s incomes, 80% of stream time is achieved through this system which translates to 1 Billion dollars of savings on customer acquisition.
Netflix uses machine learning to give the platform the ability to automate millions of decisions based on user activities. When Netflix recommends “How I Met Your Mother” because I like “Friends”, machine learning was behind that decision. This suggestion is the Netflix recommendation engine at work: it uses your past activity and returns movies and shows it thinks you would enjoy. But how is that any different from any other recommendation system?
There are multiple potential methods for creating a recommendation algorithm. A basic implementation would be the editorial method in which the platform would make suggestions based on a small number of users such as critics. Another easy one is named the simple collection method where suggestions are based on the top products across the platform. Netflix doesn’t use these methods because they do not allow personalization, nor cover the breadth of the movie catalogs and user preferences. Instead, Netflix slays the recommendation game by the use of personalized methods where movies and shows are suggested to the users who are most likely to enjoy them based on their personal ratings and people with similar interests. This way, the Netflix methodology accounts for the diversity in its audiences and its very large catalog.
Thanks to machine learning, the Netflix engineering team was able to create a “smart” platform that calculates the likelihood of a user liking a product. To understand the probability aspect of the algorithm, let’s consider this example: a utility matrix that is obtained by placing a score on the relationship between users and a movie type to predict their preferences.
Each user has seen titles on Netflix, liked some and disliked some. Based on the utility matrix, Netflix may recommend to user A a “whodunit” since they liked the thrilling plot of a horror movie they have seen recently whereas the algorithm would recommend a soft and light-hearted movie to user B because they did not like a horror movie they saw last week and give user C romance movies suggestions. And thus, each user will receive a suggestion that’s personalized for them!
Netflix interface also uses video ranking algorithms like Personalized Video Ranker (or PVR) which usually filters down the catalog by a certain criteria (violent, romance, thriller…), Top-N Video Ranker which is similar to PVR but looks only at the head of the ranking and Trending Now Ranker which captures temporal trends which Netflix deduces to be strong predictors. These rankers are used to form a row based interface where it is easy to get feedbacks as a right scroll on a row would indicate interest whilst a scroll down would indicate non-interest or irrelevance.
In a nutshell, Netflix’s recommender system collects a huge amount of data from customer behavior that is then used by machine learning algorithms in order to perfect the user experience.
Share your thoughts
Perseverance Rover: Humanity’s newest feat
February 19th, 2021 marks a new historical event in the space exploration journey, as NASA scientists rejoice.
NASA’s 2020 Mars mission successfully landed on Mars. This mission includes the Perseverance Rover along with other experiments.
While descending to touch down on the red planet, we got a close-up picture of the Perseverance rover, the first-ever high-resolution color image to be sent back by the Hazard Cameras that is most likely to become a classic photograph in the history of spaceflight.
A 360° rotation of its mast allowed the Mastcam-Z instrument to capture its first panorama, and we also got the first-ever audio recording from the red planet thanks to a microphone on the rover.
The camera system covered the whole landing process, showing the intense ride and the so-called « seven minutes of terror » descent sequence.
“Touchdown confirmed! Perseverance is safely on the surface of Mars, ready to begin seeking the signs of past life.” Said NASA engineer, Swati Mohan, during the live stream of the landing.
This landing is a considerable achievement of engineering that took multiple years to attain and was one of the main difficulties of the mission. As a matter of fact, because Mars’s air is so thin, it was more difficult to slow the spacecraft while descending, and it required a parachute and rockets, among other equipment.
Another challenge that the rover faced was the dangerous terrain it was headed to. Jezero Crater, the new home to the Perseverance Rover, was riskier than previous missions, as it presents deep pits, high cliffs, and big rocks making it harder for the rover to touch down.
However, NASA managed to design new pieces that allowed Perseverance to scan the surface and navigate around any obstacles which was a huge success.
The main goal of this mission is to hunt for the remnants of life and to study the Martian rocks up to 3.8 billion years old.
With this new source of data, the knowledge about our neighboring planet will pave the way to the future, when humans will set foot on the Martian soil.