SI 622: Evaluation of Systems and Services - Syllabus

Winter 2014

General Info

Meeting Times & Locations:
     Sec 1: Wednesdays, 1-4pm, 1255 North Quad
     Sec 2: Fridays, 8:30-11:30am, 2255 North Quad

Instructors email:
     Use this for most course business. Please put [001] or [002] at the start of your subject line in each email to indicate if you are 622 section 1 or 622 section 2. 

    Sec 1: Joyojeet Pal,
    Sec 2: Sarita Yardi Schoenebeck,

Graduate Student Instructors:
     Sec 1: Tao Dong,
     Sec 2: Manchul Han,

Office Hours & Locations:
     Sarita: Fri, 11:30am-12:30pm (NQ 3444)
     Joyojeet: Weds 4-5 PM (NQ 3445) 
     Manchul: Fri, 12:30pm - 2:30pm (NQ 3336)
     Tao: Tues, 3-5pm (NQ 3336)


Learning Objectives

Students in this course will learn to:
  • Evaluate the usability, usefulness, and acceptability of interactive software systems
  • Select among diverse evaluation methods for specific goals and types of systems
  • Write reports and give presentations that effectively communicate results of evaluations and argue for the validity and relevance of methods chosen

Course Content and Structure

This course is key component of SI's HCI curriculum and teaches skills and concepts that most HCI professionals will apply on a daily basis. In this course you learn how to evaluate the functionality and usability of an interactive system using a variety of methods:

  • Mapping the interaction flow (a diagram of the system's reaction to user input)
  • Embodying a set of target users and what they would want to do with the system
  • Comparing the features and usability of your system with competitors' systems
  • Surveying users for their opinions and experiences
  • Using checklists and “heuristics” to evaluate the system from several points of view
  • Doing user testing of typical tasks on the system
  • Incorporating new and emerging evaluation methods such as log analysis and online testing

Most of these methods will be applied to a single product that will be chosen by you and your project group. By applying various methods to a single system, you will learn not only how to do the methods but also which methods give you what answers for what kind of effort.

The course ties together learning and doing. We will give background on the methods, teach the methods, have you try them out in class, and then you will conduct the method on your own product. For many of the methods, you will turn in a report styled like the ones given to developers directly. This way you learn to write the reports in a way most useful in your future career.

The class is divided into two sections, and each meets once a week (see times above). Most of the classes will consist of a mixture of student presentatinos, lecture, exercises (with instructors consulting), and group work time (again, with instructors available to help out with various issues).

Course Materials

There are two required texts for this class:

In addition, several individual readings and websites will be made available via CTools. Lecture slides will be made available on CTools by the start of each class session.

Assignments and Grading

Assignments are products that show the results of your applying the methods taught.

Author % of Grade
Hallway Usability Test Indiv 2%
Semester Plan Group 3%
Interaction Map Group 5%
Interviews (w/Personas and Scenarios) Group 10%
Comparative Analysis Group 10%
Heuristic Evaluation Group 10%
Survey Group 15%
Usability Test Group 15%
Final Presentation Group 20%
Class Participation         Individual 10%
Total 100%

10% of your grade will be based on class participation. Participating in the class will take a number of forms, including in-class presentations, class discussions, in-class exercises, and participation in your group's activities.

Due Dates and Late Policy

  • All assignments are due one hour before class on the date listed for that assignment. 
  • Any assignment submitted more than 15 minutes after the due time (as indicated by the CTools timestamp) will be assessed a 10% penalty.
  • Each additional 24 hours of lateness will result in an additional 10% being taken off the grade for that assignment. After 5 days, the assignment will not be accepted and a grade of 0% will be entered. To spell it out:
    • Before the deadline: No penalty
    • 0:16-24:00 hours after the deadline: 10% off
    • 24:01-48:00 hours after the deadline: 20% off
    • etc., until 120:01 hours after the deadline: No credit
  • If an extension is required for legitimate reasons*, you must make your request to the instructors by email at least 24 hours before the deadline.

 * "Legitimate reasons" for late assignments include circumstances beyond your control such as client responsiveness and unusual challenges with participant recruiting. Be sure to identify these problems as soon as possible and bring them to the attention of the instructors so that we can try to find a solution that will keep you on schedule. Falling behind in 622 can be a huge headache for all involved. "Legitimate reasons" can also include illness, family emergencies, job duties, and significant emotional distress, though the group nature of the work should be able to absorb some of the impacts of these individual issues.The legitimacy of each specific instance is at the discretion of the instructors and should be brought to their attention as soon as possible. 


We will form the project groups based on input from you about who you would like to work with and who you would not like to work with. You will provide “peer evaluation” periodically, to assess the participation of the individual group members toward the assignments handed in. The end-of-semester peer assessments will be employed when calculating class participation grades.

Your first task after the groups have been formed will be to select a product (a website, device, or software application) and produce a semester plan. We will discuss the criteria for selecting an appropriate product for evaluation during the first two class meetings.


The course syllabus and schedule are hosted on this Google Site, which can be accessed via the course CTools site. The lecture slides will be made available via CTools, with links provided from the course schedule on the Google Site. Assignment descriptions and submissions will be handled via CTools. You are responsible for keeping up-to-date with the materials on CTools, as dates, assignment details, and lecture topics may change as the semester progresses. The instructors will broadcast announcements via the CTools email list whenever significant changes are made to the materials on the course site.

Academic Integrity

All assignments in this course are clearly designated at "group" or "individual" assignments. For the sole individual assignment, all submitted work must be your own, original work. For each group assignment, all submitted work must be the original work of the group. Any excerpts from the work of others (e.g., books, articles, web pages) must be clearly identified as a quotation, and a proper citation provided. You are expected to understand what plagiarism is and how to avoid it. If you are uncertain about what the boundaries are, you must educate yourself. and Purdue's Online Writing Lab provide excellent materials that can help you avoid trouble in 622 and elsewhere. Any violation of the School's policy on Academic and Professional Integrity (stated in the Master's and Doctoral Student Handbooks) will result in severe penalties, which might range from failing an assignment, to failing a course, to being expelled from the program, at the discretion of the instructor and the Associate Dean for Academic Affairs.

Accommodations for Students with Disabilities

If you think you need an accommodation for a disability, please let me know at your earliest convenience. Some aspects of this course, the assignments, the in-class activities, and the way we teach may be modified to facilitate your participation and progress. As soon as you make me aware of your needs, we can work with the Office of Services for Students with Disabilities (SSD) to help us determine appropriate accommodations. SSD (734-763-3000; sswd/) typically recommends accommodations through a Verified Individualized Services and Accommodations (VISA) form. I will treat any information you provide as private and confidential.