Skip to main navigation Skip to search Skip to main content

Automation bias in action: eye tracking of humans reading screening mammograms with and without AI prompts

  • Adnan G Taib
  • , George J W Partridge
  • , Peter Phillips
  • , Charles Maxwell-Armstrong
  • , Xin Chen
  • , Solveig S H Hofvind
  • , Jonathan J James
  • , Yan Chen
  • University of Nottingham
  • Nottingham University Hospitals NHS Trust
  • Norwegian Institute of Public Health

Research output: Contribution to journal › Journal Article › peer-review

Abstract

Background: Incorrect artificial intelligence (AI) suggestions can lead to automation bias; however, their impact on medical image interpretation is underresearched.

Purpose: To assess how incorrect AI suggestions influence the diagnostic accuracy, read times, and visual search behavior of readers interpreting screening mammograms.

Materials and Methods: In this retrospective multireader paired study conducted between September 2024 and February 2025, 10 National Health Service Breast Screening Programme mammography readers evaluated a test set of two-view mammography screening examinations. The test set included true-positive (TP), false-negative (FN), false-positive (FP), and true-negative (TN) AI suggestions, verified by 3 years of follow-up or histopathologic analysis. In round 1, readers interpreted cases without AI. In round 2, conducted 6 weeks later, a commercially available AI tool was used as decision support, displaying prompts with a region score of 10 or higher (scale, 0-100). Eye-tracking cameras recorded readers' fixations-maintained gaze-over specific image areas. Wilcoxon signed rank tests were used for paired comparisons between rounds, and Kruskal-Wallis tests compared cases with different AI outcomes.

Results: The test set ( n = 60) included cases with 26 TP, 14 FN, 14 FP, and six TN AI suggestions. Median reader sensitivity was lower for cases with FN AI suggestions when reading cases with AI (39%) compared with unassisted reading (71%; P = .002). Reader specificity was higher for cases with FP AI suggestions (39% vs 21%; P = .004). A greater number of visible (TP and FP) AI prompts led to longer median read times, from 25 seconds (zero prompts) to 34 seconds (four or more prompts) ( P = .001). Readers fixated less when reviewing cancer cases that AI failed to detect (FN suggestions) compared with unassisted reading (0.44 vs 0.47 fixations per second; P = .03). Shorter fixation durations were observed when readers interpreted cases with FP AI suggestions compared with unassisted reading (0.54 vs 0.56 second; P = .001).

Conclusion: Incorrect AI suggestions influenced both reader accuracy and visual search behaviors during mammography interpretation. The greatest negative impact was observed with FN AI suggestions; therefore, AI thresholds should be calibrated accordingly.

Original languageEnglish
Pages (from-to)e252590
JournalRadiology
Volume320
Issue number1
DOIs
Publication statusPublished - 14 Jul 2026

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

  1. SDG 3 - Good Health and Well-being
    SDG 3 Good Health and Well-being

Keywords

  • Humans
  • Mammography/methods
  • Female
  • Retrospective Studies
  • Artificial Intelligence
  • Breast Neoplasms/diagnostic imaging
  • Eye-Tracking Technology
  • Automation
  • Sensitivity and Specificity
  • Bias
  • Middle Aged
  • Intelligent Systems

Fingerprint

Dive into the research topics of 'Automation bias in action: eye tracking of humans reading screening mammograms with and without AI prompts'. Together they form a unique fingerprint.

Cite this