KLM - Keystroke Level Model


Eric J. Fimbel - Research results and data sets - home page     back to skilled users 

Citation: Fimbel, E.J. (2009) Skilled Computer Users. Available at http://skilledusers.googlepages.com/. Last retrieved : (mm/dd/yyyy) 




Introduction


"(The KLM predicts) the time it takes an expert user to perform a given task on a given computer. The model is based on counting keystrokes and other low-level operations, including the user's mental preparations and the system's response" (Card, Newell, Moran, 1980, p.396)

The KLM is currently the simplest quantitative model used in human computer interaction (see review of tools and models in Ivory & Hearst, 2001).

Urban legend. It is often said that the KLM derives from the GOMS models developed by the same authors (Card, Newell, Moran, 1983). Only the authors can say whether this is true or not. However, the dates of publication do not match.

We now present a recipy to use KLM, then an example and finally the scope of the model. It will be clearer at the end.


How it works


This recipy is based on a subset of the original KLM (Card et al. 1980). It uses heuristics from CRITIQUE, a tool to automate KLM modeling (Hudson, John, Knudsen, Byrne, 1999).

1 - Start from an interface. You can use a prototype. You can even use paper specifications.

2 - Select a set of tasks of interest (the most common, the most critical, whatever)

3 - An analyst (or yourself) examines the interface and finds 'the fastest ways' to execute the tasks.

The 'fastest ways' are called optimal strategies. They are generally defined at a coarse level, e.g., 'fill field A first, then go to next field with TAB, then ...'

4 - Convert the optimal strategies into sequences of basic operations (keystroke-level operations). The operations are for now:

K for keypresses and for mouse clicks. One K for each key and/or combination of keys (e.g., ctrl+c). One K for each click or double click.

P for pointing movements, i.e., displacements of mouse towards a determined location (e.g., a button)

5 - Complete the sequences with two operations that leave no trace on the entry devices:

H (homing) for hand displacements between mouse and keyboard. Place a H for each transition between mouse and keyboard. For instance click - keypress becomes K H K.

M (mental preparation) for mental activities required by the interaction, e.g., read a short menu, make a choice.. Place a M every time that :

the user starts working with a widget (a field to fill, a button to click..). For instance 'press tab to go to the next field' becomes K M

the user switches the interaction mode from mouse to keyboard and vice versa. 

For instance 'click a text field and start typing' becomes P K M H .... (the order of H and M is unimportant)

For instance, there is a M in the middle of a sequence of Ks whenever the user types some text, then does a different operation, like copy-paste.

The sequences prepared in this way are called here reference sequences.

6 - Count the operations of the reference sequences.

For each sequence, you will have a 4-uple (Nh, Nk, Nm, Np )

7 - Determine the predicted execution time of each sequence from the estimates of Table 1.

ET(task) = ET( reference sequence ) = Nh * Th + Nk * Tk + Nm * Tm + Np * Tp


Table 1. Keystroke level operations and estimates of execution times in milliseconds.

 operation descriptionestimate
 comment
 Hhand displacementTh =400 
 Kkey press or click
Tk = 200 or
Tk = 280
for skilled typists.
for slow typists.
 Mmental preparation
Tm = 1350

 P pointing movement
Tp = 1100

Note. There is a also a virtual operation called R(.) that represents the delays of response of the system. If you use it, you must define its duration.

We recommend avoiding the R operation: it rests generality to your sequence, because it depends on the operating system, the hardware, the network traffic, etc.

8 - If you really want a unique index of efficiency, weight the predicted execution times of the reference sequences on the set of tasks.

Index = sum( task i * weight i ) / sum ( weight i )


Example


Task-  The user enters login and password and clicks and/or key-press the OK button. 

The user starts with the hand on the mouse, the mouse elsewhere on the screen.

Figure 1. A login window.

1 - Let us consider 4 possible strategies.

usual - point-and-click field login, type "user", press tab, type "passwd", point-and-click ok

mouse - point-and-click field login, type "user", point-and-click field password, type "passwd", point-and-click ok

keyboard - point-and-click field login, type "user", press tab, type "passwd", press tab and CR.

disorder - point-and-click field password, type "passwd", press shift+tab, type "user", 
point-and-click ok.


2 - Make your mind: what is the optimal strategy?

usual, mouse, keyboard, disorder?


3 - Let us estimate the execution times with KLM

usual : sequence PKM HKKKK KM KKKKKK HPMK, predicted duration 9.65s

 stepdescriptionsub-sequence  
1point and click field loginPKM
2type "user"  HKKKK
3press tabKM
4type "passwd"KKKKKK 
5point-and-click okHPMK

sequence: PKM HKKKK KM KKKKKK HP MC

operationnumberestimateduration
H400800
K13200 2600 
M31350 4050 
P21100 2200 
total  9650ms


mouse : sequence PKM HKKKK HPKM HKKKKKK HPMK, predicted duration 11.5s


 stepdescriptionsub-sequence  
1point and click field loginPKM
2type "user"  HKKKK
3point-and-click field passwordHPKM
4type "passwd"HKKKKKK 
5point-and click okHPMK

sequence: PKM HKKKK HPKM HKKKKKK HPMK

operationnumberestimateduration
H44001600
K13200 2600 
M31350 4050 
P31100 3300 
total  11550ms


keyboard : sequence PKM HKKKK KM KKKKKK KMK, predicted duration 8.35s

 stepdescriptionsub-sequence  
1point and click field loginPKM
2type "user"  HKKKK
3press tabKM
4type "passwd"KKKKKK 
5press tab and crKMK

sequence: PKM HKKKK KM KKKKKK KMK

operationnumberestimateduration
H1400400
K14200 2800 
M31350 4050 
P11100 1100
total  8350ms



disorder: sequence PKM HKKKKKK KM KKKK HPMK, predicted duration 9.65s

point-and-click field password, type "passwd", press shift+tab, type "login", 
point-and-click ok.

 stepdescriptionsub-sequence  
1point and click field passwordPKM
2type "passwd"  HKKKKKK
3press shift+tabKM
4type "user"KKKK 
5point-and-click okHPMK

sequence: PKM HKKKK KM KKKKKK HP MC

operationnumberestimateduration
H400800
K13200 2600 
M31350 4050 
P21100 2200 
total  9650ms


4 - finally, the optimal strategy was keyboard and the measure of efficiency of your interface is 8.35s.

Remark 1. the disordered and usual strategies are in fact equivalent. The compound shift+tab and the larger pointing distance make no difference in theory and only a small difference in practice

... and both are within 20% of the optimal time, i.e., the margin of error of KLM.

Remark 2. The difference between the worst strategy (mouse) and the optimal strategy (keyboard) is in theory 3.2 seconds, i.e. 28% but in practice, the figures may be quite closer.


Personal observation. When an expert user employs a non-optimal strategy, e.g., mouse or  disorder, he or she will in fact be close from the optimal time and markedly below the prediction of KLM.

..try it yourself.


Scope of KLM


KLM provides a numerical measure of the efficiency of your interface for a set of reference tasks.

This measure is the average execution time of users that are experienced with the tasks and the interface (expert users).

The measure is a best case scenario: there is no error and no loss of time due to the task itself (counter-example: redacting a manuscript requires more thinking than transcribing).

The measure is for a standard GUI interface, using mouse and keyboard.

According to the literature, the error of prediction is typically below 20% (Card & al., 1980, Kieras, 1993, Ivory & Hearst, 2001). However, be aware on how this error is measured. 

Typically, it is the difference between the average performance of a group of users and the prediction of the KLM. This does not account for the variability of human performance.

From our experience in laboratory experiments, the variability may be for example between 50% and 20%. This means that on a single execution, the error of prediction may be 70%.


Finally, several factors may affect the quality of prediction. KLM uses

1) a set of significant tasks (the most frequent, the most difficult, etc.),
2) optimal solutions for achieving these tasks, typically established by human analysts
3) standard estimates of the execution time of basic operations.

Therefore, the quality of prediction depends on the representativity of the tasks, the quality of the optimal solutions and the adequation of the standard estimates to your hardware and software and the capacities of your users.


References


Card S.K., Moran, T.P., Newell, A., 1980. The keystroke-level model for user performance time with interactive systems. Communications of the ACM, 23(7), 396-410.

Card, S.K., Moran, T.P., Newell, A., 1983. The psychology of human-computer interactions.  New Jersey: Lawrence Erlbaum. 

Hudson, S.E., John, B.E., Knudsen, K., Byrne, M.D., 1999. A tool for creating predictive performance models from user interface demonstrations. CHI letters, 1(1), 93-102.

Ivory, M.E., Hearst, M.A., 2001. The state of the art in automating usability evaluation of user interfaces. ACM Computing Surveys, 33(4), 470-516.
 
Kieras, D., 1993. Using the keystroke-level model to estimate execution times. Unpublished Report, University of Michigan. Available at http://www.pitt.edu/~cmlewis/KSM.pdf (last consulted 2009-11-19)




Copyright: (c) 2009 E.J. Fimbel, P-S Dube. This is open-access content distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.