Rui Liu

Test Overview

Object: Gopher Mafia Games (http://www.gophermafia.com/Site/)

Procedure: Two users completed the test remotely, using their own device (laptop and mobile phone). An observer welcomed them and gave a short introduction of the purpose of the test (“We are redesigning the website and in order to improve it, we need you to help us completing 5 tasks”). During the test, the observer walked them through the process and observe their behaviors through shared screen. A total of 5 tasks were presented to users, including 4 PC tasks and 1 mobile task [11]. After users finishing all 5 tasks, the observer asked if users have any additional comment. Then the observer thanked users for participating the test and concluded the test.

Users:

• User 1: H [1], second-year grad student in the U.K, limited experience with board game. [2]
• User 2: Y [1], first-year grad student in the U.S, board game lover. [2]

Test Report

“When will there be an upcoming singles tournament?” [3]

User 1: H took a glance at the home page and clicked the “special event” tab. The result page shows the special event calendar for September. There is no singles tournament on this page. H then clicked “October” which locates at the bottom right of the page. She found a singles tournament on Oct. 12th, from 11am to 6 pm(Warhammer 40K). [4]

User 2: Y typed “singles tournament” into the search box which is next to menu bar. It leaded her to a result page with 8 results. She clicked one of them and found a singles tournament on Oct. 12th, from 11am to 6 pm(Warhammer 40K). [4]

Observer Note: Since this is the first task, I was able to collect some information about users’ first impression. As soon as H opened the website, she said “Oh this webpage is so scary”. [5] As shown above (Figure 1), the entire website uses black as its background color. This design might have negative influence on users and decrease their willingness to visit the site. For the redesign, a brighter color might give users a better first impression. [10] Talking about the task itself, it is surprising that 2 users approached the task in 2 different ways. [7] This implies that the menu bar is located and presented properly so that users are able to notice and get use of it quickly. To be more specific, menu bar is located at the center of home page and search box is represented by a commonly used icon(Figure 1. Menu bar). [8] Y’s behavior exemplifies an important role of usability, “Don’t make me think!”(Krug S, 2005). [9] Compared with search bar, the “special event” tab definitely requires more effort for a new user. In this case, Y does not have to put effort on reading the menu bar in order to look for something she needs. Unlike what we previously expected, neither of the users tried to look at the “weekly event” section which locates at the right of home page(Figure 1. Weekly Event). [6] Two possible reasons are, first, the section is not located at a place that is easy for users to notice. Second, events under that section are not presented with full name(“Warhammer 40K”) and have no detailed information. It might be hard for users to understand. [8]

“Find the return policy for Gopher Mafia Games.” [3]

User 1: H looked at the menu bar and did not find what she initially expected. So she scrolled down to the bottom of the page and, again, failed to find anything. Then she went back to menu bar and clicked “FAQ”. In the FAQ page, H found a link that addresses the return policy of this store. [4]

User 2: When the task was given to Y, she said “What is Gopher Mafia Games?” and tried to search for it using search box. I stopped her and reminded her that Gopher Mafia is the name for this website. Y then typed “return policy” in the search box instead. The result page contains 2 links that are related to store policies. Y clicked the one named “What is the Gopher Mafia Games Return Policy?” and found out the answer. [4]

Observer Note: It is really surprising that Y did not know the name of this website (Gopher Mafia Games). [7] One possible explanation is that the observer did not give her enough background information during the introduction. In addition, it is rare that we visit a website without knowing its name. Due to this reason, me, as the observer, stopped her from doing so and offered some explanation. But I do think my action is proper since her behavior might reflect the fact that users are skimming the website most times instead of reading(Krug S, 2005). [9] The design of the logo, which is also black-and-white design, might not be obvious enough for users to notice. [8] During the redesign process, the change of background may help to address this problem. [10] In addition, for this task, neither of the users thought about the “FAQ” tab at first. [6] Y continued in her style which is the search bar and H scrolled down. H said she did that because for most shopping sites, return policy is located at the bottom of the website. We did not realize that users might simply treat Gophor Mafia as a shopping site and will apply the common structure for shopping sites on it. H’s assumption can actually be very representative since the task is to look for return policy. [5]

“What is the address of Gopher Mafia Store.” [3]

User 1 & User 2: Both scrolled down directly after receiving the task. [4]

Observer Note: I am surprised by how easy this task was completed. [7] They did not go for the “contact us” tab and scrolled down directly. [6] If only H had no trouble finding the address, the reason may be that she already noticed that when she scrolled down for the previous task. But both of the users did so, I assume the reason to be that that most people have a fixed assumption that address should locate at the bottom of a webpage. That fixed assumption is so strong that they do not prioritize any other options. [8] Although this task was an easy one, I did noticed that when reporting their findings, Y and H showed some kind of uncertainty. [5] They said “Is that the right one?” Y said her uncertainty is due to the word “Contact Info” which does not equal to “address” in her opinion(Figure 2). To minimize such confusions, “Contact Info” can be moved to the phone and email section. Then add an “Address” title for the actual address. [10]

“You are a new Magic the Gathering card collector and player, find out whether you can buy and sell cards for your collection at this store.” [3]

User 1 & 2: Both users completed the following actions: scrolled up and went back to the FAQ page. Then clicked the sub-menu named “Magic The Gathering”(Figure 3). Two links about this game were returned and the first one called “Do you buy Magic the Gathering singles and/or collections?” is the correct link to look for. [4]

Observer Note: It is interesting that as the test continues, users’ actions become more alike. [5] More importantly, Y did not stick to the search box and tried something new. [6] A possible reason for that is she already got enough chance to look at the actual “content” while completing previous tasks. After learned about the site more, especially tabs in the menu bar, Y chose to go for “FAQ” tab since clicking definitely requires less effort than typing. [8] This again serves as a good example for the idea that users will only skim the website at first(Krug S, 2005). Their amount of intake for non-task-related is relatively small. [9] The surprising finding it that both users noticed the sub-menu bar which was diagnosed as a non-obvious design during the initial analyze of webpage. [7] Since users can actively notice and use it, the redesigned webpage should keep the sub-menu and maybe refine its color/design. [10]

Mobile Task:[11] “Find out how a personal event is scheduled.” [3]

To start the mobile test, users were instructed to open the website on their mobile devices. Both users were using iPhone. Since they were not able to share screen on iPhone, I asked them to speak out what they are thinking about and which button they clicked.

User 1: H clicked the icon with 3 short lines (Figure 4) and chose “FAQ” from the menu bar. She then clicked the “Events” sub-menu and find the “How do I schedule an event?” link among the 3 links returned. [4]

User 2: Same with H, Y pulled out menu bar using the icon. Then she clicked the “How to” sub-menu and reached the same link with H. [4]

Observer Note: It is interesting to see how their choice became different again. [5] Although both of them clicked “FAQ” but then they chose different sub-groups. That might because the description of this task contains both keywords, “how” and “event”. [8] It is possible for users to prioritize words differently so for the redesign, the structure of sub-menu should be preserved so that users can find what they want regardless of their way to interpret a sentence. [10] I was surprised that neither of them had trouble finding the menu bar icon since the position of it is not very noticeable. [6, 7] Maybe it is because the icon is colored red which creates contrast with the black background. [8] Color contrast is a key rule which is mentioned during lecture.[9] Although this black and red contrast seems good enough for the 2 users, it might be even better to have more contrasted colors like white and red for the redesign. [10]