This report summarizes what we evaluated in our prototype, what we learned from the planned evaluation process, and what changes we would prioritize in the next revision. Because this was an early-stage prototype, the purpose of the report was not to prove final effectiveness, but to identify whether the design was moving in the right direction and what still needed improvement.
This report is written for the design team and instructor as an internal evaluation document. Its purpose is to support revision of the current prototype and guide future improvements before fuller implementation.
The main purpose of this evaluation was to examine whether the prototype could support more structured reasoning when learners judged online information. Since the prototype is still in development, the evaluation also aimed to identify design issues that might make the experience confusing, overloaded, or difficult to use.
Our report focused on both learning and design. We were not only interested in whether participants could make better judgments after using the prototype, but also in whether the design itself was clear enough to support that process.
After using the prototype, do learners show more structured reasoning when judging an online post?
Do learners use the Who / Evidence / Motive routine in their explanations?
Which parts of the prototype feel unclear, overloaded, or difficult to follow?
How well do the role cards and tool cards support learner thinking during the experience?
The instruction being evaluated was a simplified but complete clickable prototype of our learning design, created in Figma. The prototype was based on a 40-minute teacher-led classroom lesson for students in grades 4–6, and its main goal was to help learners build a simple and repeatable routine for judging online information.
The prototype included the following sequence: a start page, role selection, role card / thinking prompt, guided post investigation, three-step check, fast feed challenge, sorting challenge, exit check, and badge/result page. The core routine used throughout the experience was: Who posted it? What is the evidence? What is the motive?
Methods
We used a combination of pre-task judgment, prototype walkthrough with think-aloud, observation, post-task judgment, and a short follow-up interview. This combination allowed us to examine both learner reasoning and design clarity at the same time.
Participants
Because we did not have access to students in grades 4–6 at this stage, we used 3 classmates as proxy participants. We recognize that adult peers are not equivalent to our target learners, but they were still useful for testing the overall flow of the prototype and for providing clearer verbal feedback during think-aloud.
Data Sources
The main materials used in this evaluation were the clickable Figma prototype, short pre-task and post-task online posts, observation notes, and follow-up interview questions. Participants were asked whether they would trust, ignore, or verify posts, and to explain why.
Overall, the evaluation suggested that the prototype is moving in a promising direction, but it also revealed several areas that still need improvement.
Finding 1: Reasoning became somewhat more structured
Participants were more likely to mention source, evidence, or motive in their post-task judgments than in their initial responses. This suggests that the three-step routine may be helping learners organize their thinking more clearly.
Finding 2: Some prototype elements were clearer than others
The general lesson goal and final decision task were relatively easy to understand, but some participants hesitated when interpreting the role cards and tool cards. This suggests that the support materials may still need simplification or a more obvious connection to the task.
Finding 3: Design revisions are still needed
Participants pointed out places where wording could be shorter, instructions could be more explicit, and screen transitions could be smoother. These comments were especially useful because they showed not only what learners thought, but also where the design itself still created friction.
Based on the findings, we would prioritize the following revisions:
Simplify the wording on role cards and tool cards.
Some participants were unsure how to use them quickly, so the text should be shortened and made more direct.
Make the connection between the cards and the three-step routine clearer.
The support materials should more explicitly guide learners toward source, evidence, and motive.
Improve transitions between prototype screens.
Some steps felt slightly disconnected, so the overall flow would benefit from clearer cues and smoother progression.
Continue testing with target-age learners if possible.
Peer participants were useful at this stage, but future testing with actual grades 4–6 learners will be important for confirming age-appropriateness and usability.
Refine the prototype before any higher-stakes evaluation.
This round was most useful as formative evaluation, helping us surface issues before fuller implementation.