Alternating Least Square Method

ALS is Matrix Factorization Algorithm. Matrix Factorization decomposes a large matrix into products of matrices.

R = U * V

For example in recommendation systems, let us consider R as a matrix of User (Rows) and Ratings (Columns). Matrix factorization will allow us to discover the latent features that define the interactions between User and Ratings. In other words, ALS uncovers the latent features.

R is m*n matrix

U is m*r matrix - User and Feature matrix

V is r*n matrix - Item and Feature Matrix

So the model will allow U and V giving a low dimension feature space. Also, space efficient for large data sets.

Now we have two unknown variables U and V. Therefore, we will use an alternating least squares approach with regularization to determine them.

we first estimate P using U and then estimate U by using P. After enough number of iterations, we are aiming to reach a convergence point where either the matrices U and P are no longer changing or the change is quite small.

Root Mean Squared Error (RMSE)

The square root of the mean/average of the square of all of the error.

The use of RMSE is very common and it makes an excellent general purpose error metric for numerical predictions.

Compared to the similar Mean Absolute Error, RMSE amplifies and severely punishes large errors.

Evaluation: Our mean of RMSE = 0.6057

We tried running the algorithm for different values of lambda, features and number of iterations. And the optimal RMSE was obtained with the below configuration:

• lambda = 0.001
• iteration = 10
• features = 10

Some of the results for different values of features and lambda are as below -

References:

1. http://cs229.stanford.edu/proj2014/Christopher%20Aberger,%20Recommender.pdf
2. https://bugra.github.io/work/notes/2014-04-19/alternating-least-squares-method-for-collaborative-filtering/
3. https://en.wikiversity.org/wiki/Least-Squares_Method
4. https://stanford.edu/~rezab/classes/cme323/S15/notes/lec14.pdf