Educational Technology

Research Tools
CoderAssist
CLuster And RepAir tool for introductory programming assignments
Papers at Learning at Scale 2014
Papers and slides at NIPS Workshop on Data Driven Education (2013)
Testing student solutions
- Stephen H. Edwards, Zalia Shams, and Craig Estep. Adaptively identifying non-terminating code when testing student programs. SIGCSE 2014. [pdf]
- Matheus Gaudencio, Ayla Dantas, and Dalton D.S. Guerrero. Can computers compare student code solutions as well as teachers?. SIGCSE 2014. [pdf]
Gaming
- Sarah Esper, Samantha R. Wood, Stephen R. Foster, Sorin Lerner, and William G. Griswold. 2014. Codespells: how to design quests to teach java concepts. J. Comput. Sci. Coll. 29, 4 (April 2014), 114-122. [pdf]
- Allison G. Martínez-Arocho, Philip Sheridan Buffum, and Kristy Elizabeth Boyer. Developing a game-based learning curriculum for "Big Data" in middle school (abstract only). SIGCSE 2014. [pdf]
- Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2013. Steering user behavior with badges. WWW 2013. [pdf]
Khan academy
- Briana B. Morrison and Betsy DiSalvo. Khan academy gamifies computer science. SIGCSE 2014. [pdf]
Papers by Stanford TLTL group
Papers on the project on Multimodal Learning Analytics
Papers by MIT ALFA group
- MoocViz: A Large Scale, Open Access, Collaborative Data Analytics Framework for MOOCs , Franck Dernoncourt,,Choung Do, Sherif Halawa, Una-May O'Reilly, Colin Taylor, Kalyan Veeramachaneni and Sherwin Wu, DDE@NIPS 2013: Data Directed Education. Workshop website.
- Analyzing Millions of Submissions to Help MOOC instructors Understand Problem Solving , Fang Han, Kalyan Veeramachaneni and Una-May O'Reilly, DDE@NIPS 2013: Data Directed Education. Workshop website.
- MOOCdb: Developing Data Standards for MOOC Datascience, Kalyan Veeramachaneeni, Zachary A. Pardos, Una-May O'Reilly, MOOCShop at Artificial Intelligence in Education, 2013. Also available from website of workshop, here.
Papers by Chinmay Kulkarni (Google Scholar)
- Scaling Short-answer Grading by Combining Peer Assessment with Algorithmic Scoring. Chinmay Kulkarni, Richard Socher, Michael S. Bernstein, Scott R. Klemmer. Learning at Scale 2014. [pdf]
Papers by Chris Piech (Google Scholar)
- Scalable Homework Search A. Nguyen, C. Piech, J. Huang, L. Guibas Camera Ready for WWW '14 [pdf][slides]
- Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC. J Huang, C Piech, A Nguyen, L Guibas AIED 2013 Workshops Proceedings Volume, 25. [pdf]
- Tuning Peer Grading C. Piech, J. Huang, Z. Chen, C. Do, A. Ng, D. Koller Proceedings of the 6th International Conference on Educational Data Mining, Memphis, Tennessee. 2013 [pdf]
- Deconstructing Disengagment in MOOCs R. Kizilcec, C. Piech, E. Schneider Proceedings of the 3rd International Conference on Learning Analytics and Knowledge, Leuven, Belgium. 2013 [pdf]
- Modeling How Students Learn To Program C. Piech, M. Sahami, D. Koller, S. Cooper, P. Blikstein Proceedings of the 43rd ACM Technical Symposium on Computer Science Education, Raleigh, USA. 2012 [pdf]
Papers by Sumit Gulwani
Overview: Example-Based Learning in Computer-Aided STEM Education, To appear in CACM, MSR Technical Report 2013, Sumit Gulwani. [abstract |pdf |bibtex |ppt slides]
Automated Feedback Generation for Introductory Programming Assignments. Rishabh Singh, Sumit Gulwani, Armando Solar-Lezama PLDI 2013 [pdf]
Synthesizing Geometry Constructions, PLDI 2011, Sumit Gulwani, Vijay Korthikanti, Ashish Tiwari. [abstract |pdf |ps |ppt slides]
Automatically Generating Algebra Problems, AAAI 2012, Rohit Singh, Sumit Gulwani, Sriram Rajamani. [abstract |pdf |full-version(pdf) |ppt slides]
A Trace-based Framework for Analyzing and Synthesizing Educational Progressions, CHI 2013, Erik Andersen, Sumit Gulwani, Zoran Popovic. [abstract |pdf]
Automatically Generating Problems and Solutions for Natural Deduction, IJCAI 2013, Umair Ahmed, Sumit Gulwani, Amey Karkare. [abstract |pdf]
Automated Grading of DFA constructions, IJCAI 2013, Rajeev Alur, Loris D'Antoni, Sumit Gulwani, Dileep Kini, Mahesh Viswanathan [abstract |pdf |full-version(pdf)]
Program Synthesis, Marktoberdorf Summer School, Aug 2013
- Part 1: From Verification to Synthesis (ppt)
- Part 2: End-User Programming using Examples and Natural language (ppt)
- Part 3: Computer-aided Education (ppt)
Synthesis for Computer-aided Education, [ppt], ExCAPE Summer School, June 2013
Synthesis for Intelligent Tutoring Systems [ppt |video], ExCAPE Webinar Series, Jan 2013
End User Programming and Intelligent Tutoring Systems [ppt], Distinguished Lecture Series at UC-Berkeley, Fall 2012
Hint Generation
- Barry Peddycord, Andrew Hicks, Tiffany Barnes. Generating Hints for Programming Problems Using Intermediate Output. EDM 2014.
Problem Solving Process
- Krisztina Tóth, Heiko Rölke, Samuel Greiff, Sascha Wüstenberg. Discovering Students’ Complex Problem Solving Strategies in Educational Assessment. EDM 2014.
MOOC
Papers by Chris Piech
Mika ̈ el Mayer and Viktor Kuncak. Game Programming by Demonstration. SPLASH Onward!. 2013. [PDF]

Papers by Petri Ihantola
- M Vujošević-Janičić, M Nikolić, D Tošić, V Kuncak. Software Verification and Graph Similarity for Automated Evaluation of Students' Assignments. Information and Software, 2012 [PDF]
- Ville Karavirta, Juha Helminen, and Petri Ihantola. A Mobile Learning Application for Parsons Problems with Automatic Feedback. In Koli Calling 2012. [PDF]
- Juha Helminen, Petri Ihantola, Ville Karavirta, and Lauri Malmi. How Do Students Solve Parsons Programming Problems? – An Analysis of Interaction Traces. In ICER 2012. [PDF]
- Werner Dietl, Stephanie Dietzel, Michael D. Ernst, Nathaniel Mote, Brian Walker, Seth Cooper, Timothy Pavlik and Zoran Popović. Verification Games: Making Verification Fun. In FTfJP 2012. [PDF]PDF,project homepage,video.
- Seth Cooper, Firas Khatib, Ilya Makedon, Hao Lu, Janos Barbero, David Baker, James Fogarty, Zoran Popovic, and Foldit players. Analysis of Social Gameplay Macrosin the Foldit Cookbook. In FDG 2011. [PDF] project homepage
- Nathaniel Rossol,Irene Cheng,Mrinal Mandal. A Workflow Based Process Visual Analyzer (ProVisZer) for Teaching and Learning. Advances in Visual Computing 2010. [Springer]
- Hsiao, I., Brusilovsky, P. and Sosnovsky, S. Web-based Parameterized Questions for Object-Oriented Programming. E-Learn 2008. [PDF]
- Brusilovsky, P. and Sosnovsky, S. Individualized Exercises for Self-Assessment of Programming Knowledge: An Evaluation of QuizPACK, ACM Journal on Educational Resources in Computing, 2005. [ACM Lib] Other QuizPACK papers
- Brusilovsky, P., Sosnovsky, S. and Shcherbinina, O. QuizGuide: Increasing the Educational Value of Individualized Self-Assessment Quizzes with Adaptive Navigation Support, E-Learn 2004. [PDF] Other QuizGuide Papers

Related Resources
Learn computer programming and Web design: Resources
Book: Reality Is Broken: Why Games Make Us Better and How They Can Change the World
- General
  - Non-CS: Duolingo
  - Non-CS: Freerice
  - General: GetKahoot
- CodingGame
- CS Unplugged
- Code Academy
  - Turing's Craft: Web-based interactive programming exercise system for intro programming classes in Python, Java, C++, C, JavaScript, C#, VB and SQL.
- Coding In The Classroom: 10 Tools Students Can Use To Design Apps & Video Games
- CodeEval
- Parsons' Puzzles
  - Parsons' Puzzles
- Open source js-parsons
- Papers on Parsons' Papers
- Parameterized Exercises/Adaptive Recommendation of Exercises
  - QuizPACK/QuizJET. QuizJET is a system serves quizzes as a self-assessment Java Evaluation Tool. It's mainly used to assess students' knowledge in Java Programming Language. QuizJET randomly generates a question parameter, creates a presentation of the parameterized question in a Web-based quiz, compares student's input to the correct answer which QuizJET runs the parameterized code "behind the stage", and records the results into a server-side database.
  - QuizGuide. QuizGuide, is an adaptive system that helps students in selecting most relevant quizzes for self-assessment of C knowledge. Quizzes are assigned to topics and adaptively annotated, to show which topics are currently important and which require further work
- Learning Data Structures and Algorithms
  - TRAKLA2. The system provides algorithm simulation exercises that can be automatically graded. The grading is based on comparison between the learner made simulation sequence and a sequence produced by an actual algorithm.
- Small Programming Exercises
  - [Code] Game at Mind Factory
  - Game web
- Pex for Fun
- Rubeque
- BetterProgrammers
- Code Forces
- Reddit Daily Programmer
  - ViLLE. Visualization assignments combine program visualization with multiple choice and array questions. In program code sorting assignments the students are asked to re-arrange rows of a program for it to perform the given task. In Coding assignment students writes a program (or part of it) by given specifications. To learn more, see the features page for complete listing.
  - Coding Bat. Coding problems in Java and Python. A problem is similar to Pex4Fun's coding duel except that the unit tests used to determine a student solution to be correct or not are a fixed set of unit tests specified by the problem creater. There a solution could be shown by clicking a button. Here are some coding bat answers.
  - CloudCoder (open source). An open-source web-based programming exercise system. It is designed to make it easy for instructors of introductory programming courses to assign short exercises to students for skills development and assessment. Currently, exercises in C/C++, Java, Python, and Ruby are supported.
  - CodeWrite. Students author their own programming exercises, practice answering exercises created by their classmates, and can evaluate and discuss all code that is written.
  - 4clojure.
- Rosetta Code
  - FLOP. a Free Laboratory Of Programming.
  - Programmer's Machine.
  - TuringsCraft. Web-based interactive programming exercise system for intro programming classes in Python, Java, C++, C, JavaScript, C#, VB and SQL.
  - Project Euler: A set of probems (391 as of July 2012) that have answers that can be typed in a text box; such answers are compared with the golden answers to determine whether you solve them or not. Problems are parameterizable.
  - IBM Ponder This Monthly Puzzles. Not necessarily programming related.
  - Sphere Online Judges (SPOJ). A problemset archive, online judge and contest hosting service accepting solutions in many languages. Correctness judging is based on a fixed set of test cases given by the problem creater or via manual judging. See tutorials here.
  - CodeChef. Similar to SPOJ: Correctness judging is based on a fixed set of test cases given by the problem creater or via manual judging. See FAQs here.
  - Leetcode Interview Question Online Judge.
  - Programming Puzzles & Code Golf. Correctness judging is based on a fixed set of test cases given by the problem creater. See FAQs here.
  - Programmer Puzzles.
- Cocode.co
  - Daily Programmers.
  - Python Challenge. It is a set of riddles that require a little bit of Python programming to be solved. The solutions are entered by changing the address of the page (URL).
  - Rankk. Rankk is about solving challenges, discovering and learning new knowledge.
  - Programming Praxis. No judging is provided. People just post solutions and problem creaters could post sample solutions.
  - Bring Your Own Code.
  - Ruby Quiz. A weekly programming challenge for Ruby programmers in the spirit of the Perl Quiz of the Week.
  - Smash the Stack. Security challenges.
  - hacker.org Challenges. A series of puzzles, tricks, tests, and brainteasers designed to probe the depths your hacking skills.
  - Google Code Jams.
  - Career Up Interview Questions.
  - Prolog Problems.
- Project-Type Programming Exercises
  - Code Kata. Simple, artificial exercises allowing to experiment and learn without the pressure of a production environment.
  - Top Coder.
  - ACM ICPC Problem Sets.
  - UVA ACM ICPC Online Judge.
  - PKU ACM ICPC Online Judge.
- Timus Online Judge
- Multiple-Choice Programming Exercises
  - IndiaBIX C# Programming Questions and Answers categorized by topics (including other Q/A exercise along with C/C++/Java exercises). Answers can be displayed on demand.
  - CS Multiple Choice Questions Bank.
- Programming Tutorials
  - Dot Net Perls.
  - Inscure Programming by Examples in C.
  - Rosetta Code. It includes solutions to the same task in as many different languages.
  - Algorithmist.
- Grading
- Web-CAT
- Resources
- Stackexchange