Abstract
Objective: The present study replicated and extended prior findings of suboptimal automation use in a signal detection task, benchmarking automation-aided performance to the predictions of several statistical models of collaborative decision making. Background: Though automated decision aids can assist human operators to perform complex tasks, operators often use the aids suboptimally, achieving performance lower than statistically ideal. Method: Participants performed a simulated security screening task requiring them to judge whether a target (a knife) was present or absent in a series of colored X-ray images of passenger baggage. They completed the task both with and without assistance from a 93%-reliable automated decision aid that provided a binary text diagnosis. A series of three experiments varied task characteristics including the timing of the aid’s judgment relative to the raw stimuli, target certainty, and target prevalence. Results and Conclusion: Automation-aided performance fell closest to the predictions of the most suboptimal model under consideration, one which assumes the participant defers to the aid’s diagnosis with a probability of 50%. Performance was similar across experiments. Application: Results suggest that human operators’ performance when undertaking a naturalistic search task falls far short of optimal and far lower than prior findings using an abstract signal detection task.
Original language | English |
---|---|
Pages (from-to) | 945-961 |
Number of pages | 17 |
Journal | Human Factors |
Volume | 64 |
Issue number | 6 |
Early online date | 2021 |
DOIs | |
Publication status | Published - Sept 2022 |
Keywords
- decision-making strategies
- human–automation interaction
- naturalistic visual search
- signal detection theory