(1) Is this game playable as a whole? Are there any loopholes or dead ends? (Logic)
(2) Are the interfaces intuitive to navigate? (UI)
(3) Are the game rules communicated clearly? (Rules)
(4) Are the learning contents understandable? (Content)
(5) Does it create a playful experience and desired emotions? (UX)
(6) What are the potential drop-off reasons? (UX)
Since the research questions cover behavioral, cognitive, and attitudinal aspects of a player's experience, and the prototype is not a full prototype but only contains one sample scenario and one possible path, I adopted a moderated playtesting approach, combining observation, think aloud and interview methods, to obtain as much information as possible. Observation focuses on the smoothness of navigating in the game (RQ 1 & 2); Think aloud prompts are mainly targeted to the player’s understanding of the game rules and learning takeaways (RQ 3 & 4), and the post-game interview is for the overall impression of the experience (RQ 5 & 6).
There is a myriad of possible play paths and they are hard to test through brute force without a powerful algorithm. I decided to use case analysis approach to test the rules, that is to test the best path the game rules try to encourage, the worse paths to be punished, some random paths with legit moves, and some path with failed tasks, then compare the results to see if the scores can differentiate between each other and if it causes a quick death.
I made a simulation spreadsheet to run the different cases, documented the results, adjusted the rules bit by bit, and annotated the iteration process on a Mural board. I also invited a peer with game experience to review the logic of the rules.
The scenario design is the meat of the whole project. It requires not only clarity and simplicity to digest in a short period of time, but also the accuracy of the messages and the nuances of the choices to ensure that the choices are credible and meaningful but not too obvious. I want to test if my scenario writing meets these requirements.
I developed a scenario writing template and 18 topics with some initial ideas, filled out the template with a narrative challenge description, three options (an optimal solution, the worst solution, and a conditional pass solution), and animation scripts for each consequence. I invited 3 peers with a learning sciences background to talk through the ideas and examine the appropriateness. Further scrutiny by subject matter experts is needed in the future.