Well, it turns out that there are a number of ways of doing that by using one of the methods below:Alternating Least Squares(ALS)Stochastic Gradient Descent(SGD)Singular Value Decomposition(SVD)3.
Hyperparameter OptimisationBefore tuning the parameters we need to pick up an evaluation metric.
A popular evaluation metric for recommenders is Precision at K which looks at the top k recommendations and calculates what proportion of those recommendations were actually relevant to a user.
Therefore, our goal is to find the parameters that give the best precision at K or any other evaluation metric that one wants to optimize.
Once the parameters are found, we can re-train our model to get our predicted ratings and we can use these results to generate our recommendations.
4.
Post ProcessingWe can then sort all of the predicted ratings and get the top N recommendations for the user.
We would also want to exclude or filter out items that a user has already interacted with before.
In the case of movies, there is no point in recommending a movie that a user has previously watched or disliked earlier.
5.
EvaluationWe have already covered this before but let’s talk in a bit more detail here.
The best way to evaluate any recommender system is to test it out in the wild.
Techniques like A/B testing is the best since one can get actual feedback from real users.
However, if that’s not possible, then we have to resort to some offline evaluation.
In traditional machine learning, we split our original dataset to create a training set and a validation set.
This, however, doesn’t work for recommender models since the model won’t work if we train all of our data on a separate user population and validate it on another.
So for recommenders, we actually mask some of the known ratings in the matrix randomly.
We then predict these masked ratings through machine learning and then compare the predicted rating with the actual rating.
Evaluating recommenders OfflineEarlier we talked about Precision as an evaluation metric.
Here are some of the others that can be used.
Python LibrariesA number of Python libraries are available that are specifically created for recommendation purposes.
Here are the most popular ones:Surprise: A Python scikit building and analyzing recommender systems.
Implicit: Fast Python Collaborative Filtering for Implicit Datasets.
LightFM: Python implementation of a number of popular recommendation algorithms for both implicit and explicit feedback.
pyspark.
mlib.
recommendation: Apache Spark’s Machine Learning API.
ConclusionIn this article, we discussed the importance of recommendations in a way of narrowing down our choices.
We also walked through the process of designing and building a recommendation system pipeline.
Python actually makes this process simpler by giving access to a host of specialised libraries for the purpose.
Try using one to build your own personalised recommendation engine.
ReferencesAn introduction to recommendation systems in machine learningHow to Design and Build a Recommendation System Pipeline in Python (Jill Cates).