Recommendation Engine built using Spark and Python

Here is an example of trying a few different combinations of ranks, lambdas and iteration counts: $ ../bin/spark-submit recommend.py train ratings.dat –ranks=8,9,10 –lambdas=0.31,0.32,0.33 –iterations=3 The best model was trained with: Rank: 10 Lambda: 0.320000 Iterations: 3 RMSE on test set: 0.931992 Getting Recommendations The recommendation engine needs to know your opinion on films which have been rated by a lot of other users..The metrics command shows which films have the largest number of user ratings: $ ../bin/spark-submit recommend.py metrics ratings.dat movies.dat 10 most rated films: 3,428 #2858 American Beauty (1999) 2,991 #260 Star Wars: Episode IV – A New Hope (1977) 2,990 #1196 Star Wars: Episode V – The Empire Strikes Back (1980) 2,883 #1210 Star Wars: Episode VI – Return of the Jedi (1983) 2,672 #480 Jurassic Park (1993) 2,653 #2028 Saving Private Ryan (1998) 2,649 #589 Terminator 2: Judgment Day (1991) 2,590 #2571 Matrix, The (1999) 2,583 #1270 Back to the Future (1985) 2,578 #593 Silence of the Lambs, The (1991) Ive picked 5 films which have a lot of ratings and added a parameter to the recommend command which let you rate each of them..1 is a poor film, 5 is the best and 0 if you havent seen it..The films are American Beauty (1999), Jurassic Park (1993), Terminator 2: Judgement Day (1991), The Matrix (1999) and Back to the Future (1985)..The following parameter rates them 5, 3, 5, 5 and 4 accordingly: –ratings=5.0,3.0,5.0,5.0,4.0 So with the ratings, rank, lambda and iterations picked you can now see which films are recommended viewing..$ ../bin/spark-submit recommend.py recommend ratings.dat movies.dat –ratings=5.0,3.0,5.0,5.0,4.0 –rank=10 –lambda=0.32 –iteration=3 His Girl Friday (1940) New Jersey Drive (1995) Breakfast at Tiffany's (1961) Halloween 5: The Revenge of Michael Myers (1989) Just the Ticket (1999) I'll Be Home For Christmas (1998) Goya in Bordeaux (Goya en Bodeos) (1999) For the Moment (1994) Thomas and the Magic Railroad (2000) Message in a Bottle (1999). More details

Leave a Reply