91.568 Seminar in HCI, Spring 2013

To-Do list based on 1 April 2013 class

By Wednesday 3 April, fill out the entire Heuristic Groups spreadsheet.

By Friday, 5 April, send all the HUS spreadsheet information to Christine.

By Monday, 8 April, have the Fleiss' Kappa statistic completed. To do the Kappa statistic, use the Nielsen column and the Coding column. Compare: Nielsen column for each individual to see if there is agreement. Coding column for each individual to see if there is agreement. If the Coding column = Composite, check to see that at least one of the Nielsen heuristics is the same among all of the coders, regardless of whether the individual put that heuristic in the Nielsen, Nielsen 2, or Nielsen 3 column. If the Fleiss' Kappa statistics work out, then analyze using bar charts, etc. and provide results on Monday. If there are problems with the Kappa, the analysis team will let the class know right away, and we will rework the definitions and recode.

Everyone, when rereading their own sets of heuristics, should add to the Summary table of heuristic sets in the column that Neal will add, whether the heuristics are replacements for Nielsen's or are meant to be used in addition to Nielsen's.

"Our" SUS is now called "HUS"!

...for "Heuristic Usability Scoring." The version we worked on in class is available for download on the left hand side of this page under "course materials."

Notes from class held 18 March 2013

Comparison of specialized heuristics to Nielsen's heuristics

One of the analyses will be to determine the similarity between specialized heuristics and Nielsen's. Multiple coders will categorize each heuristic, and we will calculate Fleiss' Kappa statistic to determine inter-rater reliability. The proposed categories are as follows:

• • Derivative: related to but narrower than a specific Nielsen heuristic

  • Expanded: related to but broader than a Nielsen heuristic

  • Composite: a heuristic that combines the intent of two or more of Nielsen’s heuristics.

  • Identical: a heuristic is the same as Nielsen’s heuristic.

  • Unrelated: a heuristic is not covered by any of Nielsen’s heuristics.

Analysis: For each set of specialized heuristics, we will determine the percentage of heuristics in that set that fall into each of these categories.

Question: Is there “a missing heuristic”? Will one of the heuristics in the “other” category rise to the level of a more general heuristic?

Major tasks that are left to do:

• Divergence from Nielsen’s heuristics (data collection and analysis as described above) – all except Merve and Jing. Neal will develop a spreadsheet that will enable data collection and send it out to the class by next Monday.

• Table construction and analysis of patterns from template characteristics – Neal will construct a table for the survey document, and Efe, Luke, and Elad will analyze it.

• Timeline -- Christine

• SUS – Christine will determine how many papers each person will read, and will determine the SUS scale itself. All will contribute SUS scores, except Merve and Jing will only contribute scores on papers they have already read.

• Review what has been written so far

How do we justify which heuristic sets that we chose?

Specialized heuristics are only useful if they can be found, so nine people looked for as many specialized heuristic sets as possible, doing Google searches and noting papers about heuristics that were referenced in other papers. We will also show the variety of sources that we ended up finding using them, such as conference papers and journal articles from a wide variety of subfields. Also we will show that we have a large variety of applications that the heuristic apply to.

Other tasks/deadlines

Jill will email Holly to ask if we can have class members access multi-touch tables this week.--DONE

Everyone should put their names at the top of each heuristic writeup.

By Thursday at 5 p.m. EDT, everyone should have their write-ups included, at which point the collection of heuristic sets will be frozen.

The new syllabus, with revisions regarding specific deadlines, has been uploaded to the bottom of this page.

Round 2 of heuristic writeups

Elad:

F. Vetere, S. Howard, S. Pedell, and S. Balbo. Walking through mobile use: Novel heuristics and their application. In Conference of the Computer-Human Interaction Special Interest Group of the Human Factors and Ergonomics Society of Australia OZCHI2003

Neal:

J. Mankoff, A. K. Dey, G. Hsieh, J. Kientz, S. Lederer, and M. Ames. Heuristic Evaluation of Ambient Displays. In Conference on Human Factors in Computing Systems - CHI'03, pages 169-176. ACM Press, 2003.

Efe:

Hasiah Mohamed @ Omar and Azizah Jaafar (2010). Heuristics Evaluation in Computer Game.

Merve:

Heuristics for Information Visualization Evaluation

Torre Zuk, Lothar Schlesier, Petra Neumann, Mark S. Hancock, Sheelagh Carpendale

Jing:

Adapting Heuristics for Notification Systems

Brandon Berry

Sean:

Heuristic Evaluation for Interactive Games within Elderly Usershttp://www.thinkmind.org/index.php?view=article&articleid=etelemed_2011_6_10_40059

Christine:

Heuristic Evaluation of Persuasive Health Technologies

Luke:

Heuristics for Video Games

Course Description

The purposes of this seminar course are: to involve students in current HCI research, to learn to critique others’ HCI research, and to learn about the breadth of research possibilities under the general umbrella of HCI. The theme of this offering of the HCI Seminar is specialized techniques for heuristic evaluation. Class members will co-author a paper documenting a review of specialized heuristic evaluation techniques and will also develop a new heuristic evaluation technique. We will validate the new technique and write a paper for submission to a conference or journal. In addition, students will be responsible for critiquing several research papers in detail and act as participants in an HCI-related experiment.

Groups

The two groups will be comprised of the following people:

Efe, Neal, Luke, Christine

Sean, Elad, Merve, Jing