Automation bias in action: how AI influences human decision making and reading behaviour during mammography interpretation

Background: To assess how incorrect AI suggestions influence the diagnostic performance and visual search behaviour of breast cancer screening readers.

Materials and methods: In this retrospective paired reader study, 10 NHS Breast Screening Programme readers (median experience: 14 years, IQR 7–25) evaluated mammograms between 2024 –2025 while being eye-tracked. In round 1 (R1), readers interpreted cases without AI. After six weeks, the readers reinterpreted the same 60 cases with AI support (Lunit) in round 2 (R2). The AI displayed prompts with a region suspicion score ≥ 10 (range 0 low –100 high). The test set was purposefully enriched with 14 (23%) FNs and 14 (23%) FPs, as well as 6 (10%) TNs and 26 (43%) AI TPs, according to pathology or 3-year follow-up. Paired comparisons between rounds used Wilcoxon signed-rank tests; Kruskal–Wallis tests were used for between-group comparisons.

Results: For AI FN cases, median human sensitivity decreased from 71% in R1 to 39% in R2 (p < 0.01). For AI FPs, specificity increased from R1 (21%) to R2 (39%, p < 0.01). Readers fixated less frequentl y when reading cancer cases that the AI failed to detect (FNs) compared to when AI was not used (R2 0.44 vs R1: 0.47 fixations/s, p = 0.03). Shorter fixations were observed when readers interpreted AI FP cases, compared to when AI was not used (R2 0.54 s vs R1 0.56 s, p < 0.01).

Conclusion: Incorrect AI prompts influence reader performance and search behaviours. FNs were associated with a decrease in performance; therefore, thresholds should be calibrated accordingly for AI-assisted reading.

(British society of breast radiology annual scientific meeting abstracts - abstract O5. BSBR Annual Scientific Meeting 2025 Brighton, UK 9–11 November 2025 https://www.delegate-reg.co.uk/bsbr2025.)