Situation Awareness

• • Process indices - These reveal that a person is actually making a situation assessment, which should result in situation awareness. Psychophysiological assessments (e.g. EEG and eye tracking) are commonly used as process indices.

  • Performance Measures - These suggest that the SA has an impact on performance. Appropriate SA would be beneficial to performance, while inadequate SA would be detrimental to performance. The performance measures used are closely related to the situation in question. Response-time, rate of success, and frequency of an appropriate response are common performance measures used.

  • Behavioral Measures - These postulate that some behaviors are the result of more or less SA. Instead of putting the emphasis on outcomes, these methods focus on how individuals come to these outcomes. These approaches can differ based on the level of SA, so a behavioral change can indicate a change in SA, though this is not always the case. Comparison between an ideal behavior and an observed behavior can indicate the level of SA.

  • Observer rating technique - An impartial observer takes on the responsibility to rate the SA of the person under study. This person is typically a peer to the person under study, as they have more experience with the task than the experimenter would have, and are able to be less biased towards the true goals of the experiment. The observer can cross-validate the appropriateness of the self-ratings by the person under study to assess SA.

References:

1. Andersen, H.H.K., and Hauland, G. (2000). Measuring team situation awareness of reactor operators during normal operation: A technical pilot study. Proceedings of the first human performance, situation awareness, and automation conference. Savannah, GA.

2. Bell, H.H., and Lyon, D.R. (2000). Using observer ratings to assess situation awareness. In M. R. Endsley and D.J. Garland (Eds), Situation Awareness Analysis and Measurement (pp. 129-146). Mahwah, NJ: Lawrence Erlbaum Associates Inc.

3. Wilson, G.F. (2000). Strategies for psychophysiological assessment of situation awareness. In M.R. Endsley, and D.J. Garland (Eds), Situation Awareness Analysis and Measurement (pp. 175-188). Mahwah, NJ: Lawrence Erlbaum Associates Inc.

• • Think aloud - Individuals verbalize what they are thinking as they perform the task. As such, this method interferes with aspects of the on-going task. It is a beneficial tool when a task analysis cannot reveal SA requirements, however it is severely limited by what individuals do not reveal. Simply ask the participant to verbalize what they are thinking while they are performing the task, and probe them for continual feedback.

  • Freezing technique - The task is paused and participants are asked to report their perception and understanding of all SA-related aspects of the task. The task is then resumed. This is a fairly intrusive technique, as participants do not resume the task in the same mindset as they did before the interruption. This method is typically used in simulators when the action can be easily paused and resumed.

    • Example Freezing Techniques include:

      • SAGAT (Situations Awareness Global Assessment Technique)

  • Real-time Probes - Questions are addressed to the person while they complete the task; it is not paused. A set of questions may be created based on the SA requirements and probed at the appropriate points during the on-going interaction. This is a less intrusive method that can be used for tasks that cannot be easily paused and resumed, but it is less exhaustive since fewer queries can be asked. SAGAT-like probes can be used as real-time probes.

References:

1. Breton, R. and Rousseau, R. (2003). The Think Aloud Technique. Situation Awareness: A Review of the Concept and its Measurement. (pp. 41-42). DREV TR- 2001-220 Defense Research Establishment Valcartier.

2. Endsley, M.R. (1988) Situation awareness global assessment technique (SAGAT). In Proceedings of the National Aerospace and Electronics Conference (NAECON). (pp.789-795). New York:IEEE.

3. Jones, D.G. and Endsley, M.R. (2000). Examining the validity of real-time probes as a metric of situation awareness. Proceedings of the 14th Triennial Congress of the International Ergonomics Association and the 44th Annual Meeting of the Human Factors and Ergonomics Society.

• • Recall Awareness Techniques - Subjective measures, like questionnaires, are presented after an individual have completed the activity. Subjective measures of SA can be highly correlated with actual performance measurements of SA. Ideally, the subjective measure of SA would be compared with an observer rating by an impartial party.

    • Example subjective questionnaires include:

      • SALIANT (Situation Awareness Linked Indicators Adapted to Novel Tasks)

      • SART (Situation Awareness Rating Technique)

  • Recall Situation Techniques - Individuals rebuild the state of a system after an activity has ended. An individual may be asked to recall where certain people were during a specific point during the event or activity. This recollection can also be compared to the actual representation of the situation. Beware: this technique is not validated as a measure of SA, but the information collected can contribute to understand SA.

References:

1. Breton, R. and Rousseau, R. (2003). The Retrospective Measurement Technique. Situation Awareness: A Review of the Concept and its Measurement. (pp. 45-49). DREV TR- 2001-220 Defense Research Establishment Valcartier.

2. Muniz, E.J., Stout, R.J., Bowers, C.A., and Salas, E. (1997). A methodology for measuring team situation awareness. Paper presented at the 18th Annual Industrial/Organizational Psychology/Organizational Behavior Graduate Student Conference. Roanoke, VA.

3. Selcon, S.J. and Taylor, R.M. (1990). Evaluation of the situational awareness rating technique (SART) as a tool for aircrew system design. In Situational Awareness in aerospace operations (AGARD-CP-478). (pp. 51-58). Neuilly-sur-Seine, France: NATO-AGARD.