I can do better than your AI: expertise and explanations

J Schaffer, J O'Donovan, J Michaelis, A Raglin… - Proceedings of the 24th …, 2019 - dl.acm.org
J Schaffer, J O'Donovan, J Michaelis, A Raglin, T Höllerer
Proceedings of the 24th International Conference on Intelligent User Interfaces, 2019dl.acm.org
Intelligent assistants, such as navigation, recommender, and expert systems, are most
helpful in situations where users lack domain knowledge. Despite this, recent research in
cognitive psychology has revealed that lower-skilled individuals may maintain a sense of
illusory superiority, which might suggest that users with the highest need for advice may be
the least likely to defer judgment. Explanation interfaces-a method for persuading users to
take a system's advice-are thought by many to be the solution for instilling trust, but do their …
Intelligent assistants, such as navigation, recommender, and expert systems, are most helpful in situations where users lack domain knowledge. Despite this, recent research in cognitive psychology has revealed that lower-skilled individuals may maintain a sense of illusory superiority, which might suggest that users with the highest need for advice may be the least likely to defer judgment. Explanation interfaces - a method for persuading users to take a system's advice - are thought by many to be the solution for instilling trust, but do their effects hold for self-assured users? To address this knowledge gap, we conducted a quantitative study (N=529) wherein participants played a binary decision-making game with help from an intelligent assistant. Participants were profiled in terms of both actual (measured) expertise and reported familiarity with the task concept. The presence of explanations, level of automation, and number of errors made by the intelligent assistant were manipulated while observing changes in user acceptance of advice. An analysis of cognitive metrics lead to three findings for research in intelligent assistants: 1) higher reported familiarity with the task simultaneously predicted more reported trust but less adherence, 2) explanations only swayed people who reported very low task familiarity, and 3) showing explanations to people who reported more task familiarity led to automation bias.
ACM Digital Library