We want to teach children about AI that learn through reinforcement learning and reward functions. Our idea is to have the user program their own AI model to play the popular game: snake. They will program it by specifying the agents reward functions (e.g when to reward, when to punish, when to do nothing, what numerical values to give each). This will be an interactive application, as the user will have the ability to program/debug it in different ways, and compare each model they train to evaluate for themselves which model performs better and why. In regard to gathering data, we can save each model that the user makes and from there, observe wether or not they were able to properly train the AI. From saving the model, we can derive it's high score, average score, number of iterations; all this we can use to evaluate how well the user did.
We plan to gather data on each individual that uses our app. We will have them enter their first name, and grade level.
Quantitive: Once we can separate the data between each user, we plan to save each AI model that the user creates (as they can create many models). From here, we could derive some data from each model that will give us information on how each AI performed (e.g. the models high score, average score, reward functions). We can maybe compare each model with an ideal model and base it off of that. Since the child will ideally create multiple models, we can then compare all of them and see if there was any progress in the child's understanding of the concept (e.g. if Model 2 performed better than Model 1). Lastly, we plan to add 3 multiple choice questions to our post survey, this will also give us quantitive data. On the technical side, we can maybe use plain text files where each file represents a user; another option is to use sqlite (lightweight database) to store the data.
Qualitative: By recording the screen while the child uses the app, we can gather some qualitative data on how the interaction went. Along with this, our 2 post survey questions will also give us more qualitative data.