The goal of this project is to offer personalized Spotify music recommendation to new users. At the beginning, we were scared by the amount of data we were dealing with, and not sure how to start using these data. We took inspiration from literatures and past works, and began to learn the logic behind recommendation systems and performed exploratory data analysis.
The first challenge we had was how to handle all these data. After consideration, we decided to use 10% of the data for training due to limited computational resources, and we were aware of the trade-off between the project manageability and model accuracy. We also decided to focus on Collaborative Filtering exclusively to maximize the use of the data we already in hands. We did gather tag data from Lastfm, but had a hard time combining the Million Playlist Dataset and the Million Song Dataset. Forcefully combining them would lose a lot of information from both dataset.
The second challenge was that we were not sure how to evaluate the model, since there is no ground truth in the provided dataset. After consulting with our TF and doing research, we came up with a validation dataset. We split this validation dataset into X and Y. We fed X into the model and compare the recommendation produced by the model to the Y, and compared how much they overlapped (R-Precision). NDCG helps to evaluate the whether more relevant songs were recommended first.
After cleaning the data, we started building with very basic Collaborative Filtering model using matrix factorization. The R-Precision was around 0.03 and NDCG was around 0.013. On average, there are 3 songs in the recommendation match the songs in the ground truth set, and it took a long time for a relevant song to show up in the recommendation. These results were not great. Thus, we introduced audio features into this model, in hope to augment the performance. We used the audio features to further select songs that are more similar to the songs in the playlist. However, the results did not improve.
One shortcoming was that the model only ran for 5 epochs. (It took hours to run 1 epoch on a MacBook). With a extremely sparse matrix like this, it would need many more epochs for its loss to be minimized. It would also be beneficial to use more data to train, but that comes with more computation that is hard to handle.
It is intuitive to think that audio features would help the model make better recommendations. When we used audio features to re-rank the recommendations, the performance did not improve at all. We hypothesize that because users often put songs from the same genre in a playlist, songs in a given playlist are then very similar in terms of audio features. For example, the most common playlists in MDP are named 'chill' and 'country'. Songs in 'chill' are more slow and low in energy, and songs in 'country' are more energetic. Thus, in our model, if we were to assume the original recommended songs were already similar in their audio features, there was no need to re-rank them using audio features anymore. Instead, it would be more help for to use audio features to select songs using a separate model. Additionally, comparing the similarity between each song pair in the dataset in real time is extremely slow. One way to overcome this is to use dynamic programming and compute all these values in advance and store them in a data structure. Also, we can also try utilize the property of K-d tree.
In order to come with a better recommendation system that takes less time train and is more efficient, we started to explore Neural Network Embeddings, which is more efficient by significantly reducing the dimension of the interaction matrix.
As shown in the Neural Network Embeddings section, the model produced some promising results, but at the same time, it still has limitations.
For our next step, we want to keep improving our model by incorporating more data, and also, find a way to tune the model so that it recommends a range of more diverse songs to the users.
Hope you enjoyed our website! If you have any questions, please email us at: