Weekly Journal

Week 1:

Week 1 involved education about the REThink program requirements, Machine Learning, and our Brain Sensing research project. We also developed our REThink Research Proposal.

Week 2:

Reproducing Rule-Learning - determine detailed requirements needed to reproduce experiment and results described in “rules-learning.pdf” using fNIRS

Elevator Speech - reviewed/commented on Pat Rahmlow's write-up: 30 second; two minute; 140 characters

Literature Reviews completed:

• Anterior Prefrontal Cortex Mediates Rule Learning in Humans ,Strange, B. A., Henson, R. N. A., Friston, K. J., & Dolan, R. J., 2001 https://drive.google.com/open?id=1ZqOyRScBVkqM4aoPlp5qLWydIgMbwRoP2DRBkLqOiEI
• Example Tracing Tutors: A New Paradigm for Intelligent Tutoring Systems, Aleven, V., McLaren, B., Sewall, J., Koedinger. K. R.
  • Notes:
    • Questions to answer (applicable to studies) didn’t lend itself to review of this paper
    • Format of this review consisted of:
      • How does CTAT compare to ASSISTments?
      • summary of CTAT as an ITS

AP CS Principles / AP CS A sharing:

• I shared my AP Table Leader (Inaugural AP CS Principles Reading June 9th - June 18th in Kansas City) experiences with other AP teachers:
  • interpretation of Fall 2016 requirements and rubric for the Create Performance task
  • common mistakes and how College Board handled
  • changes for next year Fall 2017
• will be participating in AP CS Principles Mock Reading July 8th and July 9th in Reston VA

GitHub presentation

Week 3:

Reproduce Rule-Learning Study

• Caitlin will do the coding, using our reqs doc, and as of Wed, she will base her coding on a text file that Pat and I provide with scripted stimuli, condition, and correct ans
• Generated our "pseudo" random list of 4 letter strings, for 1 exemplar. What I did is:
  • enter into an Excel spreadsheet all possibilities using the letters A, X, Q.
  • I sorted the list alphabetically so that I could remove the duplicates.
  • I then entered random #'s into another column, then locked in the random #'s (copied the column of random #'s and pasted special values(v)), then sorted on the column of locked random #'s.
  • Excel spreadsheet is called "Exemplar random.xlsx" on google drive in the "Rules Learning-Summer 2017" folder. It should be relatively easy to generate 2 more exemplars by just replacing the letters.
  • Concern: Are there too few correct examples of rules? There are 78 different variations, but only 24 were correct for any given rule. I posted this concern and discussed with Pat, suggesting we could delete some of the incorrect. Pat, more experienced than I in working with Excel, did some refinements and additional work on the list to create a data file ready for Caitin's Rule-Learning program.

Look into CTAT and compare to ASSISTments in terms of use by AirLab - what to do with the fNIRS data, specifically how to adapt what is presented to the student in CTAT

• Question: How does CTAT compare to ASSISTments
  • Installed Cognitive Tutor Authoring Tools (CTAT) for Java and CTAT for HTML and Flash, needed to install a 32-bit Java JRE on my 64-bit Windows laptop required by CTAT
  • successfully created a CTAT example-tracing tutor for fraction addition following this CTAT "Creating an example tracing tutor" Tutorial
    • Characteristics of this fraction addition tutor:
      • there are 2 solution paths, one which is preferred, along with an alternate
      • there is 1 incorrect solution path, a common error (adding the denominators) made by students who have not mastered fraction addition
      • 3 hint messages were provided for each correct solution path
      • 1 buggy message was provided
      • Skill Labels entered included skill name "Least common denominator / lcd" and skill set name "Fraction-addition"
  • Next I plan to try to create a tutor using CTAT's "example-tracing" feature to solve a multiple step equation such as: 3(2x-5) = 9. I plan to include 2 correct solution paths, 1 preferred and 1 alternate , along with at least 1 incorrect solution path. Some considerations/questions:
    • Since I used a provided Fraction Addition student interface to create my first "Fraction Addition Tutor", it's not clear how much work is involved in building a student interface from scratch, in this case a Solve multi-step equation student interface; update: I successfully created a student interface for Fraction Addition using the same screen layout used in the provided Fraction Addition student interface as a model; Conclusion: not much work once I have a model student interface to refer to, altho I would want to search for other student interfaces using screen layouts that would be appropriate
      • CTAT's MathTutor appears to have a number of student interfaces that could be used as a model when developing a new student interface for creating algebra tutors
    • Also I would prefer to pull skill names from a commonly accepted mathematics standard list of skill concept names or knowledge components
    • Although I was successful in terms of creating a tutor, I stored my files to my laptop desktop, I need to change my setup to use google drive- Resolved 7/12
    • how to create a tutor that includes 2 or more problems
    • how to deploy any tutor
      • CTAT's TutorShop might be a good candidate
• Ques: Can CTAT accept input from fNIRS and thus modify the content presented to the students?
  • Completed lit review of an article Integrating MOOCs and Intelligent Tutoring Systems: edX, GIFT, and CTAT that discusses using existing e-learning interoperability standards, namely the Learning Tools Interoperability (LTI), resulting in CTAT accepting parameter input that is used to adapt the content presented to students
  • Add update - based upon ITS meeting Wed afternoon with Erin
    • after Erin reviewed the files resulting from creating a CTAT "Example-Tracing" tutor, she stated that she felt comfortable that her lab would be able to "hack" the code, i.e. it would be feasible to use fNIRS data to adapt what a student sees in a tutor
• Ques: Can knowledge components be associated with incoming data
  • Progress as of 7/12: Successfully created a CTAT example-tracing tutor for fraction addition following the CTAT "Creating an example tracing tutor" Tutorial including manually entering Skill Labels entered included skill name "Least common denominator / lcd" and skill set name "Fraction-addition"

CTAT's MathTutor and TutorShop

• submitted a request for a developer account in TutorShop and a teacher account for MathTutor, after a couple of days with no response...
• emailed TutorShop & MathTutor's Principal Investigators Vincent Aleven, PhD and Bruce M McLaren, PhD at Carnegie Mellon University's Human-Computer Interation Institute and cc'd Dr Erin Solovey asking for contact to expedite accounts

Meetings, Presentations, ...

• Met with Cailin to finalize requirements of rule-learning study which she will code
• Met with Juan for a AirLab GitHub presentation
• Machine Learning presentation

AP CS Principles / AP CS A sharing

• I shared my AP CS Principles Mock Reading July 8th and July 9th in Reston, VA:
  • I was one of a group of AP CS Principles Create Performance Task (PT) readers who read (graded) student submitted Create PTs using a draft of the 2017-18 Create PT rubric , basically "vetting" the 2017-2018 rubric
  • changes for Fall 2017 consists of changing the 2016-2017 Create PT rubric (0 to 3 points per each of 4 items) to a binary rubric (0 or 1 point per each of 7 items

Week 4:

Reproduce Rule-Learning Study

• Using Caitlin's Experiment Checklist for n-back experiment as a guide
  • conducted the rule-learning experiment on 5 subjects from AirLab
    • all so far have fallen short of success:
      • accidental power outage
      • imotions stopped recording
      • accidental adjustment of the fNIRS sensors without recalibrating from the start
    • however altho the data was invalid, the data output from one of the experiment runs was used to input into the DB/MATLAB
  • created an Experiment Checklist for rule-learning
    • most updates have been made, specifically noted changes needed as a result of running the 5 experiment runs
    • some updates need to be made to the script that describes the rule-learning experiment

CTAT's MathTutor and TutorShop

• Obtain accounts to access Mathtutor and TutorShop
• Determine how useful Mathtutor's student interfaces are as models for creating new tutors
• Determine how to use TutorShop to deploy newly created tutors

Week 5:

Literature Reviews completed:

• Aleven, V., McLaren, B. M., Sewall, J., Van Velsen, M., Popescu, O., Demi, S., ... & Koedinger, K. R. (2016). Example-Tracing tutors: Intelligent tutor development for non-programmers. International Journal of Artificial Intelligence in Education, 26(1), 224-269.
  • Notes - CTAT advantages over ASSISTments:
    • multiple solution paths and multiple error paths per problem
    • mappings between problem steps and knowledge components
    • CTAT/TutorShop architecture allows plug in of different options for student modeling & task selection
    • fully compatible with DataShop, open repository for educational technology data sets that supports offline analysis of tutor log data
• Booth, Julie and Koedinger, Kenneth R., “Key Misconceptions in Algebraic Problem Solving.” Proceedings of the Annual Meeting of the Cognitive Science Society, 30. (2008)
  • Notes - “rule learning” parallels with “solving equations learning”
    • Solving 1-step, 2-step and multi-step equations
      • With focus on common misunderstanding of the meaning of equals or negative signs
      • are examples of authentic Algebraic learning
    • Would “solving a 1-step equation learning” have the same brain activity as “rule learning”?
      • Valuable information in an adaptive personal learning environment to improve student learning

REThink Project - Rule Learning Experiment

Completed Pilot Study experiments

Analyzing data

REThink Poster

Preliminary version, out for review

Lesson Plan - Preliminary concept

Students in AP Computer Science A / AP CS Principles will use the Pittsburgh Science of Learning Center (PSLC) DataShop public repository of research data using the DataShop analysis and reporting tools. Students will also export data from the repository for further analysis using programming and / or spreadsheet software.

Meetings, Presentations, ...

• Women in CS Presentation

CTAT's MathTutor and TutorShop

• emailed CMU's Dr. Tom Cortina asking if he could identify contacts to help expedite a request for a developer account in TutorShop and a teacher account for MathTutor, resulted in
  • Access to Mathtutor: progress as of 7/28 3:52 PM: I received an email from CMU Mathtutor support requesting that I obtain a signoff from my school district's Math Supervisor / Principal before I can obtain full teacher access (includes teachers and students in my district) to Mathtutor. I've sent the form along to my Math Supervisor for signoff - hoping to receive back from him before the end of the week-end and will then mail off to CMU
  • Access to TutorShop: progress as of 7/28 5:24 PM :Received response from CTAT-TutorShop staff, For developer access, they are asking for information about our plans, specifically:
    • ...we'll need to know a bit about your plans, for we need to balance the load on several servers and fit your project into the overall scheme of TutorShop sites. Do you have CTAT tutors all ready to deploy and test? Are they related to math or other domains? Roughly how many problems do you anticipate deploying? When do you expect real students to begin using your tutors? Can you give us any idea how many total classes and students would use the site? In particular, can you guess how many might be using the site at once? E.g., "the most we anticipate during a single school period would be 3-5 classes of 20-30 students each."
    • In order to put your CTAT tutors on the web and test them, we'd give you a content developer account on one of our test sitesOnce you've had a chance to see how your tutors work on the web, we'd figure what operational site you should use for real students. Please reply to the list mathtutor-support@lists.andrew.cmu.edu<mailto:mathtutor-support@lists.andrew.cmu.edu>. Thanks for your interest in TutorShop,