Accomplished task: - Product/Item based recommendation & ALS
What we would likely to accomplish: - Use of different data-sets and get more products in recommendation. Also we would try to make our recommendation engine as real time as possible.
Additional comments and observations
Challenge:
Generating pivot table for such a big data was one of the challenge in spark framework.
Recommendation systems was completely new topic for us, so we had to give some more time is understanding on how different algorithms are used for it.
We were not able to execute any of the Spark code on AWS EMR, so we had to work on the UNCC DSBA cluster, which was busy most of the time.
Things learnt:
We got exposed to a lot of new and interesting spark functionalities and features.(e.g. broadcast, dealing with different kind of RDDs)
Got some basic overview of how to work with Amazon Web Services.
Team work, as the task we pre-defined everything went on smoothly and we were able to achieve which we expected to.