Advanced Interaction Design (2015)

(Course last given in spring 2015)

Main Course topics: Demographics and design, User Experience (UX), Personas, Responsive Design, Design Patterns, Prototyping, wireframing using Balsamiq, Qualitative and Quantitative Evaluation of Interaction Designs, Ethical evaluation procedures, Descriptive and Inferential Statistics, SPSS data analysis, Speech and Natural Language Interfaces, Conversational Agents, Multimodal Interfaces, Human-robot Interaction design.

Assessment: 50% exam and 50% coursework. All students sit the same exam. Masters students are expected to demonstrate more extensive critical awareness and understanding of the course material in their coursework project (see below). Coursework will be marked at 80% for the final report, 10% for the presentation, and 10% on a self-reflection report about the group collaboration process (see below).

Coursework -- group project: design / analyse / develop / evaluate (see below) a novel user interface that uses multiple modalities for interaction (i.e. two or more of: graphics, speech, audio, tactile, gesture, movement, ....). For example: use speech on Android, or OpenDial or IrisTK, to design a new spoken dialogue system; use a Leap Motion or Kinect or a touch-table to design a new gesture-controlled interface; combine this with the Oculus Rift virtual reality system; use the NAO or FurHat to create a Human-robot interaction system; design a location-based mobile app that tells users (using speech) what amenities are nearby, perhaps based on a user model, or plays location-based songs or ..... be creative!

The project theme for this year is interfaces for the `smart home', see e.g. and the video links below!

Equipment that you can use:

  • MYO gesture control armband
  • Oculus Rift
  • Google Glass
  • NAO torso
  • FurHat head
  • Very large touchscreens
  • Touchtables
  • Lots of Android mobiles / tablets
  • Leap motion - fingertip / hand gesture recognition
  • Kinect (V1 and V2)
  • Eyetrackers

Your project must cover several of the following aspects:

  • design: explain and motivate possible interaction designs for your chosen interface, including user demographics, personas, user experience, design patterns. Compare and contrast at least 2 possible designs;
  • analyse: based on research literature and previous approaches relevant to your interface, write an analytical review of the main problems and research issues relevant to your design problem;
  • develop: use some of the tools listed below (or other relevant software, or your own code) to develop a working prototype of your interaction design;
  • evaluate: using either a working prototype or a functional mockup of your design(s), perform a contrastive evaluation of 2 or more possible interaction designs, using qualitative and/or quantitative methods. Analyse your results for statistical significance.

Possible group-member roles:

* = essential role (some team member(s) must do this!)

(many people will have several roles, and most projects will not have all of these roles):

  • Implementation / engineering / programming
  • Testing
  • Evaluation with users *
  • User Experience
  • Experimental Design *
  • Project Management *
  • Leadership *
  • Literature Review *
  • Document preparation /report writing *
  • Presentation *
  • Demographics
  • Heuristic evaluation
  • User consultation groups/ focus groups
  • Participatory design
  • Data analysis / statistics *
  • Graphic design
  • Information Architecture
  • Prototyping
  • Ethical Issues *
  • Overall Concept *

Deliverables: (submit all of these via email to the lecturers, with email header: Name, group name, deliverable name)

  1. Project presentation (week 9, 10% of coursework): group delivers a 10 minute presentation to the class describing your chosen design problem and your project plan and progress.
  2. Research paper (week 12, 80% of coursework): write up your group project and results in the format of a 6-page conference paper submission, using the CHI template (use either the latex template, or word template): see edited Word template attached at bottom of this page!
  3. Individual self-reflection report on your experience of the group process (week 12, 10% of coursework).

This is a short (1 page) report answering the following questions:

How did you plan and manage your own work within the group?

To what extent did you independently solve problems and take initiative within the group?

How did you take responsibility for your own and other’s work by contributing effectively and conscientiously to the work of your group?

How did you actively maintain good working relationships with group members?

Did you lead the direction of the group project or any aspect of it?

Critically reflect on your roles and responsibilities within the group, and the roles and responsibilities of the other members.

Guest lectures: NLP -Arash Eshghi; UX - Stephen Denning; HRI ?? ; visit to User Vision

Research space and equipment: Google Glass, Oculus Rift, Leap Motion, Kinect v2 dev kit, NAO robot torso, FurHat animated robot head, various eyetrackers, etc

Lectures / labs / class plan:

Tuesday classes: 14.15- 15.15 in EM 336, Friday classes 10.15-12.15 in EM 307 (for weeks 7-9 10.15 class is in MGB22)

  • Week 1:
    • Tuesday class: Introduction, multimodal user interfaces (demos); Coursework project intro (Lemon/Foster)
    • Friday class: Demographics and User Experience; Hands-On Design Task (Foster)
  • Week 2:
    • Tuesday class: group project work
    • Friday class: Personas; Heuristic Evaluation, Prototyping; lab using Balsamiq (Foster)
  • Week 3:
    • Tuesday class: group project work
    • Friday class 1: Responsive Design; Design Patterns (Foster)
    • Friday class 2 : Introduction to Natural Language Processing and Conversational Interaction Design (Eshghi/Lemon)
  • Week 4:
    • Tuesday class: group project work
    • Friday class: Qualitative Evaluation of interaction designs, NVivo, Ethics procedures (Foster)
  • Week 5:
    • Tuesday class: group project work - each group send project description by email by 5pm:
      • Project Title,
      • group name,
      • group members and their roles in the team (see possible roles listed above)
      • project main aim/ objectives,
      • target user group(s),
      • design methods,
      • implementation plans (if any),
      • evaluation methods and plans,
      • references list / related work
    • 13th Feb: Friday class: Quantitative Evaluation for Interaction Design: intro to descriptive statistics and statistical significance (Lemon)
  • Week 6:
    • Tuesday class: group project work
    • Friday: VISIT to User Vision -- 20th February - space is limited to about 15 people so *please sign up* (Lemon/Foster)
    • Friday class: Group project work (if you are not coming to User Vision)
  • Week 7:
    • Tuesday class: group project work
    • Friday class 1: Quantitative Evaluation: descriptive statistics and statistical significance; lab (MGB 22) using SPSS (Lemon)
    • Friday class 2: Inferential statistics and evaluation of interaction designs; Wizard-of-Oz data; logfile analysis; Questionnaire design (Lemon)
  • Week 8:
    • Tuesday class: group project work
    • Friday class 1: Inferential statistics and evaluation of interaction designs - Wizard-of-Oz data and logfiles; Parametric/Non-parametric data; lab (MGB 22) using SPSS (Lemon)
    • Friday class 2: Conversational Interfaces / Spoken Dialogue systems / Multimodal Interfaces / Gamification (Lemon)
  • Week 9:
    • Tuesday class: group project work
    • Friday class: Group project presentations, marking, + feedback (everyone)
  • Week 10:
    • Tuesday class: group project work
    • Friday class: Conversational Interfaces: Speech Recognition, Language Understanding, Dialogue Management and Language Generation and Speech Synthesis; lab using OpenDial or IrisTK (Keizer)
  • Week 11:
    • Tuesday class: group project work
    • Friday class: Human-Robot Interaction (Foster)
  • Week 12:
    • Tuesday class: Revision with past exam papers (Lemon / Foster)
    • Deadline: FRIDAY 3rd April: Group project report (6 pages, CHI conference-paper style, use either the latex template, or word template) - send to us both by email

Software and Tools

Cereproc: speech synthesiser

Balsamiq: wireframing / mockup tools

NVivo: software for qualitative research

NLTK: Natural Language ToolKit

CoreNLP: Stanford parsing and NLP tools

Praat: speech analysis software

ELAN: annotation tool

Android speech API: speech recognition and synthesis on Android

OpenEars: free speech recognition and synthesis on iPhone

Web speech API for Chrome

KALDI: speech recognition toolkit

Boxer: language understanding

WIT AI : API for spoken language understanding

OpenDial: dialogue system toolkit

IrisTK: multimodal dialogue system toolkit

Voice XML

SPSS: statistical analysis (this is installed on the university computers)

Sirius: open source personal assistant (like Siri)

Estimote iBeacons: micro-location and contextual awareness

Relevant videos

Interaction Lab videos: SpeechCity, JAMES, ECHOES, Parlance

Incremental spoken dialogue system: "Numbers"

Google video on speech understanding, deep neural networks

Cortana, AI, and project Adam

WIT AI : API for spoken language understanding

Stanford NLP lectures

FurHat the social robot

Jibo personal robot

Apps to play with

Google Now

Assistant (ai.api)





Videos of home automation interfaces

These might help you think about things you could do for your group projects:

Android home automation:

Kinect gesture-based controls:

Amazon Echo:

FurHat robot videos:

Home automation dialogue system from POSTECH:

iPad graphical interface:

VoxCommando Vera:

Crestron iPad interface:

OpenHab NAO robot interface (french):