Workload

Measures of workload can be classified as 1) performance-based measures, 2) subjective measures, or 3) physiological measures. Performance measures assume that increased task demands directly impact performance, while subjective measures assume that people are able to accurately perceive the level of effort required for a task and that effort is related to workload. Physiological measures assume that the level of physiological activation reflects the degree of mental workload. It is important to note that workload cannot be directly measured, and therefore all measures described are indirect assessments.

1. Performance-based measures of workload:

• Primary task performance - Capability to perform the main task; Does not take into account spare mental capacity and may be less sensitive to changes in workload at low and intermediate levels.

• Secondary task performance - Capability to perform a second task in addition to the main task; Accounts for spare mental capacity, but the secondary task may disrupt primary task performance. It is critical that a secondary task use the same resources as the primary task to allow for accurate measurement. See Wickens (2008) for an explanation of the four-dimensional multiple resource model. Ogden et al. (1979) report from their literature analysis that the most frequent categories of secondary tasks used are choice reaction time, memory, monitoring, and tracking.

2. Subjective measures of workload:

• NASA Task Load Index (TLX) - uses six dimensions on a 100-point scale to assess mental workload: mental demand, physical demand, temporal demand, performance, effort, and frustration

• Subjective Workload Assessment Technique (SWAT) - uses three levels: (1) low, (2) medium, and (3) high, for each of three dimensions of time load, mental effort load, and psychological stress load

• Workload Profile (WP) - asks the subjects to provide the proportion of attentional resources used across 8 dimensions: perceptual/central processing, response selection and execution, spatial, processing, verbal processing, visual processing, auditory processing, manual output, and speech output

3. Physiological measures of workload:

• Heart rate/Heart rate variability

• Blood pressure

• Electroencephalogram (EEG)

• Skin conductance response

• Eye measures (e.g., pupil dilation, blink rate)

When selecting a workload measurement it is important to consider factors such as:

• Sensitivity - the power of the measure to detect changes in task demands

• Selectivity/Validity - accurately assesses the intended variable (i.e., mental workload) by responding only to changes in cognitive demands, and not to changes in other variables (e.g., physical demands).

• Diagnosticity - the ability to identify reasons for variations in workload

• Intrusiveness - does not interfere with the task being evaluated

• Reliability - consistent across items, over time, and between researchers

• Implementation requirements - e.g., time limitations, instruments, and software required for collection and analysis

In some cases, it may be useful to combine multiple measures for comparison of results.

References

1. Ogden, G. D., Levine, J. M., & Eisner, E. J. (1979). Measurement of Workload by Secondary Tasks. Human Factors, 21(5), 529–548. https://doi.org/10.1177/001872087902100502

2. Rubio, S., Díaz, E., Martín, J., & Puente, J. M. (2004). Evaluation of subjective mental workload: A comparison of SWAT, NASA-TLX, and workload profile methods. Applied Psychology: An International Review, 53(1), 61–86. https://doi.org/10.1111/j.1464-0597.2004.00161.x

3. Veltman, J. A. & Gaillard, A. W. K. (1998). Physiological workload reactions to increasing levels of task difficulty. Ergonomics, 41(5), 656-669, https://doi.org/10.1080/001401398186829

4. Wickens, C. D. (2008). Multiple Resources and Mental Workload. Human Factors, 50(3), 449–455. https://doi.org/10.1518/001872008X288394