The goal of this assignment is to get experience performing a heuristic evaluation of a real user interface, and to provide useful feedback to your classmates on their interface designs. Note that there is an in-class discussion of recommendations that you write up and submit.
Feedback sessions are scheduled explicitly during class time. You may also schedule more than the 40 minute allotments outside of class if needed. The 40 minute in-class eval slots can't be adjusted trivially because they would generally cause someone else to miss something else.
see the Design Refinement Phase Checklist for your pairing.
In this assignment, you will act as a consultant to another group in the class. Assume that they would like some outside assistance in finding problems with their prototype interface.
This assignment will proceed in three steps.
Individual Evaluation:
Visit the online prototype provided by the team whose work you are evaluating (available to you by the date specified in the Design Refinement Checklist). Read their instructions and explore their prototype. Apply Nielsen's heuristics (or a different set of heuristics if you think a different set is more appropriate and your evaluation group has agreed) to the interface. You should do this in two passes. In the first pass, read the group's notes/instructions and then try out their prototype (i.e., run it); walk through their task scenarios. In the second pass, go through the interface (using the scenarios again if appropriate) and look for and note HE violations.
You should concentrate on the interface the group has designed, not only on what has been implemented. Do not focus on features that are missing, but would clearly be added (e.g., "there should be help on this screen... and this screen..." -- if it is a globally missing feature you can report it once). We want to focus on evaluating what they have designed so far. You should also bear in mind whatever the team may tell you (or whatever their site may suggest) about their target audience.
Use this severity rating scheme:
0. Don't think this is a usability problem. 1. Cosmetic problem 2. Minor usability problem 3. Major usability problem; important to fix 4. Usability catastrophe; imperative to fixproblem # [heuristic # and name] (severity rating) description of problem and reasoning why you think this violates the heuristic5. [H4 Consistency & Standards] (Severity 2) The interface used the string "Save" on the first screen for saving the user's file, but used the string "Write file" on the second screen. Users may be confused by this different terminology for the same function. 6. [H3 User Control & Freedom] (Severity 3) The interface brings the user into a set of preference screens when they select "New User", but doesn't allow the user out of the dialog until they fill out all 4 screens. There is no way to cancel from any of the screens if a user came into the first screen by accident. .....You may also want to include some notes about things that you found that you thought were good, but there is not a formal format for these. The individual evaluation writeup should take no more than two printed pages. You shouldn't spend more than two hours on this part of the assignment.
Group Evaluation:
The group report should begin with an overview evaluation of the project. This is where you can discuss aspects you found particularly compelling as well as summarize the major issues found.
The central part of the group evaluation should compile all usability violations, eliminating duplicates. You may find it convenient to present this information in a table. Make a note of how many evaluators found each error. Decide on the appropriate severity measure for each violation; negotiate among yourselves and use an average if you can't come to agreement. List the total number of violations in each severity category and each heuristic type.
The group report should close with some overall recommendations for improving the user interface. The group report should take no more than 6 printed pages, including a paragraph or so at the beginning and at the end and a table in the middle summarizing violations. Think about how best to make this document useful for the team to whom you are delivering the report. Attach the individual reports to the group report as an appendix.
You shouldn't spend more than an hour and a half on this part of the assignment.
here is an example from a previous team that can guide what your final heuristic evaluation summary can look like: http://hfid.olin.edu/sa2012/s_engr3220-eris/designrefinement/hfid_eris_heuristic_evaluation.pdf
Individual Reflection:
In this assignment, you will write up take-aways after completing the entire Heuristic Evaluation Process. Please answer the prompts below in the text box for submission on Canvas.
1) To what degree did your individual severity ratings of usability issues match the scores of others on your team? Why do you think that was the case? Do you believe that your team used effective tools for addressing differences and coming to consensus? If not, what could be improved?
2) From your perspective, did the feedback you gave the other team and the report you receive seem to flag similar problems? Were the severity of issues similar? What do you think might have led to these differences or similarities?
3) Did you find Nielson's set of heuristics adequate for characterizing the feedback that you wanted to give? What did you find yourself writing in yourself or wanting to say? Did you personally save a lot of your reactions for the in-person discussion or did you feel that your writing could capture your recommendations?
Please submit your individual and group reports and reflection to Canvas