Embedded Files
Purpose:The purpose of our first round was to have proxy users participate in a blind play-test to analyze sentiments about individual elements and the overall design for the board game mockup. Due to the virtual environment of this project, this testing was conducted with college students serving as proxy users, instead of children within our target age range of eight to twelve years old. Through this testing we wanted to assess two factors: (1) which gameplay elements were intuitive to proxy users, and (2) the overall level of enjoyment experienced by the proxy users while play-testing.
Test Methodology:Two team members conducted one large testing session with the board game mockup on May 1st, during which eight Northwestern college students interacted with the board game mockup. In order to make the proxy users as similar to the direct users as possible, only students with a maximum of one year of high school chemistry were selected to test the mockup. The board game mockup contained one large game board, a molecule building kit containing pieces representing carbon, oxygen, and hydrogen atoms, two molecule answer keys, four cardboard player pieces, and a set of eight molecule cards.
Test Methodology:Two team members conducted one large testing session with the board game mockup on May 1st, during which eight Northwestern college students interacted with the board game mockup. In order to make the proxy users as similar to the direct users as possible, only students with a maximum of one year of high school chemistry were selected to test the mockup. The board game mockup contained one large game board, a molecule building kit containing pieces representing carbon, oxygen, and hydrogen atoms, two molecule answer keys, four cardboard player pieces, and a set of eight molecule cards.
Figure 16: Closeup on Gameboard
Figure 17: Overall Board Game Setup
The players were given basic gameplay instruction, which included an explanation of the rules of the game and an identification of which molecule pieces corresponded to which atom. For the test, the proxy users played through one full round of the game, and the entire session lasted about thirty minutes.
For context, a written description of a sample turn taken by the proxy users during the game is included in this methodology. A player would try a card from the main pile, which would list a molecule containing some combination of carbon, hydrogen, and oxygen atoms. The player would then attempt to build that molecule using pieces from the molecule building kit. Once they finished their build, the game master would check their molecule against the answer key. If correct, the player would move the indicated amount of spaces forward. If incorrect, the player would move back a space. The card would be placed in the discard pile, and the next player would begin their turn.
Results:
The following table summarizes the user proxy results. The questions, as well as the answers to both the pre and post tests can be found below.
Test Questions:
Rate your ability to name and create molecules on a scale of 1 to 5. 1 being most confident.
Name Five Molecules.
a. Metric for assessing performance: number of correct molecules listed
What is the molecular formula for water?
a. Metric for assessing performance: correct answer (C) vs incorrect answer (I)
What is the molecular formula for methane?
Additional Post-Test Questions:
How likely are you to play this game again on a scale of 1 to 5. 1 being most likely.
How much did you enjoy playing this game on a scale from 1 to 5. (1 being loved, 5 being hated)
Answers shown in table below. The order of the answers always corresponds to the eight participants in numerical order.
Table 4: Board Game Mockup Feedback
Analysis, Conclusions, and Limitations
Analysis of Results: The board game mockup showed some potential for increasing the chemistry knowledge of the proxy users, but was met with a lukewarm reception in terms of gameplay and overall entertainment.
Chemistry Knowledge: The board game mockup showed some potential for increasing the chemistry knowledge of the users. Almost all users showed improvement in the second, third, and fourth questions on the knowledge test. The first question of the test also demonstrated that six of the eight proxy users believed that their general confidence in naming and creating molecules had improved by playing the game.
Entertainment Factor: The answers to questions 5 and 6 on the post-test indicate a mediocre enjoyment and willingness to replay the mockup. From observation and interviews, the proxy users complained about the difficulty of building abstract molecules, as well as the tendency of molecule cards to repeat. The players very much enjoyed the physical aspect of the game and being able to build the molecules with their own hands, and there was some friendly competition apparent between the users. The users liked the fun designs on the game board.
Shortcomings of the Mockups: There were two large, glaring issues with this mockup that emerged as a result of user testing. Firstly, there was too large a range in the building difficulty of the molecules, with both extremes being disliked by users. Molecules like oxygen gas were too simple for the users to build and provided no sense of success, while molecules like acetic acid were too frustrating and difficult for the players to enjoy building. In addition, there were not enough molecule cards as the deck of molecule cards was recycled twice throughout the game, which resulted in familiarity and a lack of variety in the gameplay.
Conclusion: The results suggest that this mockup is mediocre at improving the chemistry knowledge of the users, but is lacking in terms of user experience, specifically ease of play and overall enjoyment of the game. There are two design challenges that must be addressed if future iterations of this mockup are to be successful.
Limitations: The overall conclusions drawn from this mockup testing are primarily limited by the nature of the participants being proxy users. Although the participants were fitted and ensured to have limited chemistry knowledge, all of the proxy users had taken at least one chemistry class in high school. As a result, the baseline familiarity of the proxy users will be greater than that of our target age range. When extrapolating from this testing, the improvement shown in the pre and post test must be evaluated within the context of age. If this mockup is selected as the final design, testing with a more sophisticated prototype must be completed with users from the target age range.
Analysis of Results: The board game mockup showed some potential for increasing the chemistry knowledge of the proxy users, but was met with a lukewarm reception in terms of gameplay and overall entertainment.
Chemistry Knowledge: The board game mockup showed some potential for increasing the chemistry knowledge of the users. Almost all users showed improvement in the second, third, and fourth questions on the knowledge test. The first question of the test also demonstrated that six of the eight proxy users believed that their general confidence in naming and creating molecules had improved by playing the game.
Entertainment Factor: The answers to questions 5 and 6 on the post-test indicate a mediocre enjoyment and willingness to replay the mockup. From observation and interviews, the proxy users complained about the difficulty of building abstract molecules, as well as the tendency of molecule cards to repeat. The players very much enjoyed the physical aspect of the game and being able to build the molecules with their own hands, and there was some friendly competition apparent between the users. The users liked the fun designs on the game board.
Shortcomings of the Mockups: There were two large, glaring issues with this mockup that emerged as a result of user testing. Firstly, there was too large a range in the building difficulty of the molecules, with both extremes being disliked by users. Molecules like oxygen gas were too simple for the users to build and provided no sense of success, while molecules like acetic acid were too frustrating and difficult for the players to enjoy building. In addition, there were not enough molecule cards as the deck of molecule cards was recycled twice throughout the game, which resulted in familiarity and a lack of variety in the gameplay.
Conclusion: The results suggest that this mockup is mediocre at improving the chemistry knowledge of the users, but is lacking in terms of user experience, specifically ease of play and overall enjoyment of the game. There are two design challenges that must be addressed if future iterations of this mockup are to be successful.
Limitations: The overall conclusions drawn from this mockup testing are primarily limited by the nature of the participants being proxy users. Although the participants were fitted and ensured to have limited chemistry knowledge, all of the proxy users had taken at least one chemistry class in high school. As a result, the baseline familiarity of the proxy users will be greater than that of our target age range. When extrapolating from this testing, the improvement shown in the pre and post test must be evaluated within the context of age. If this mockup is selected as the final design, testing with a more sophisticated prototype must be completed with users from the target age range.
Page updated
Report abuse