It seems smart, but it acts stupid: Development of trust in ai advice in a repeated legal decision-making task

PK Kahr, G Rooks, MC Willemsen… - Proceedings of the 28th …, 2023 - dl.acm.org
Proceedings of the 28th International Conference on Intelligent User Interfaces, 2023dl.acm.org
Humans increasingly interact with AI systems, and successful interactions rely on individuals
trusting such systems (when appropriate). Considering that trust is fragile and often cannot
be restored quickly, we focus on how trust develops over time in a human-AI-interaction
scenario. In a 2x2 between-subject experiment, we test how model accuracy (high vs. low)
and type of explanation (human-like vs. not) affect trust in AI over time. We study a complex
decision-making task in which individuals estimate jail time for 20 criminal law cases with AI …
Humans increasingly interact with AI systems, and successful interactions rely on individuals trusting such systems (when appropriate). Considering that trust is fragile and often cannot be restored quickly, we focus on how trust develops over time in a human-AI-interaction scenario. In a 2x2 between-subject experiment, we test how model accuracy (high vs. low) and type of explanation (human-like vs. not) affect trust in AI over time. We study a complex decision-making task in which individuals estimate jail time for 20 criminal law cases with AI advice. Results show that trust is significantly higher for high-accuracy models. Also, behavioral trust does not decline, and subjective trust even increases significantly with high accuracy. Human-like explanations did not generally affect trust but boosted trust in high-accuracy models.
ACM Digital Library