Extend the Spark DSL.
Recall the Spark DSL for MapReduce programming presented by Matei in Lecture 2. It turns out that you can build a DSL on top of Spark, for example, for programming a class of machine learning algorithms. In addition to the Scala implementation of Spark, a Python version will soon be available, so you can embed your DSL in either of these host languages.
Final Project >