Week 6

For our deliverable this week we are supposed to discuss one thing we have learned about research. I've learned that it really is a LOT of work to do research well. A ton of preparation is needed even to answer a seemingly simple question such as grade prediction. The data pre-processing stage in particular has been highly time consuming as it turns out real world data is a lot trickier to work with compared to a toy project/class assignment.

We managed to sort through the zyBook data and extract some features from it so that we are now fully utilizing all of our available data. There is a ton of zyBook data and so the processing to extract features takes a long time each time I want to run the model. Thus, I did a bunch of code reorganization to make it easy to automate batches of predictions. I ran some experiments using varying amounts of data from the course trying to predict final exam scores. The results are below. It's interesting that I seem to be getting the best predictions after 9-10 weeks and then the results actually get worse using more data from later in the course. On the right you can see a plot of the regression coefficient associated with each variable from one of my predictions. This allows me to gain some insight into how the model is working.

In other news, I finally went to the Fairfax farmers market for the first time on Sunday where I got momo (Nepali dumplings) and some fresh peaches.