Overreliance on AI: Literature Review

Overreliance on AI occurs when users start accepting incorrect AI outputs. This can lead to issues and errors that can ultimately make people lose trust in AI systems. This report explains what overreliance on AI is, how it happens, and how we can mitigate it. An important goal of AI system design is to empower users to develop appropriate reliance on AI. This is important given that policymakers and practitioners call for greater human oversight—making users the last line of defense against AI failures. This report shows how and why overreliance on AI makes it difficult for users to meaningfully leverage the strengths of AI systems and to oversee their weaknesses. Based on a literature review of ~60 papers from different research areas, this report provides a detailed overview of how overreliance on AI happens, how to measure overreliance, what its consequences are, and how we can minimize its negative effects.

See also our 2024 research synthesis about Appropriate reliance on Generative AI

Cite as: Samir Passi & Mihaela Vorvoreanu. 2022. Overreliance on AI: Literature Review. Microsoft Technical Report MSR-TR-2022-12. Microsoft Corporation.