research-article

Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making

Authors:

Snehal Prabhudesai,

Nikola BanovicAuthors Info & Claims

IUI '23: Proceedings of the 28th International Conference on Intelligent User Interfaces

Pages 379 - 396

https://doi.org/10.1145/3581641.3584033

Published: 27 March 2023 Publication History

Abstract

Decision Support Systems (DSS) based on Machine Learning (ML) often aim to assist lay decision-makers, who are not math-savvy, in making high-stakes decisions. However, existing ML-based DSS are not always transparent about the probabilistic nature of ML predictions and how uncertain each prediction is. This lack of transparency could give lay decision-makers a false sense of reliability. Growing calls for AI transparency have led to increasing efforts to quantify and communicate model uncertainty. However, there are still gaps in knowledge regarding how and why the decision-makers utilize ML uncertainty information in their decision process. Here, we conducted a qualitative, think-aloud user study with 17 lay decision-makers who interacted with three different DSS: 1) interactive visualization, 2) DSS based on an ML model that provides predictions without uncertainty information, and 3) the same DSS with uncertainty information. Our qualitative analysis found that communicating uncertainty about ML predictions forced participants to slow down and think analytically about their decisions. This in turn made participants more vigilant, resulting in reduction in over-reliance on ML-based DSS. Our work contributes empirical knowledge on how lay decision-makers perceive, interpret, and make use of uncertainty information when interacting with DSS. Such foundational knowledge informs the design of future ML-based DSS that embrace transparent uncertainty communication.

References

[1]

Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for Human-AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300233

Digital Library

[2]

Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

[3]

Matthew Arnold, Rachel K. E. Bellamy, Michael Hind, Stephanie Houde, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Darrell Reimer, Alexandra Olteanu, David Piorkowski, Jason Tsay, and Kush R. Varshney. 2018. FactSheets: Increasing Trust in AI Services through Supplier’s Declarations of Conformity. https://doi.org/10.48550/ARXIV.1808.07261

[4]

Syed Z. Arshad, Jianlong Zhou, Constant Bridon, Fang Chen, and Yang Wang. 2015. Investigating User Confidence for Uncertainty Presentation in Predictive Decision Making. In Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction (Parkville, VIC, Australia) (OzCHI ’15). Association for Computing Machinery, New York, NY, USA, 352–360. https://doi.org/10.1145/2838739.2838753

Digital Library

[5]

Nikola Banovic, Zhuoran Yang, Aditya Ramesh, and Alice Liu. 2023. Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust. Proc. ACM Hum.-Comput. Interact. 7, CSCW1, Article 27 (apr 2023), 17 pages. https://doi.org/10.1145/3579460

Digital Library

[6]

Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S. Weld, Walter S. Lasecki, and Eric Horvitz. 2019. Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence(AAAI’19/IAAI’19/EAAI’19). AAAI Press, Honolulu, Hawaii, USA, Article 300, 9 pages. https://doi.org/10.1609/aaai.v33i01.33012429

Digital Library

[7]

Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the Whole Exceed Its Parts? The Effect of AI Explanations on Complementary Team Performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 81, 16 pages. https://doi.org/10.1145/3411764.3445717

Digital Library

[8]

Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusnezov. 2019. The need for uncertainty quantification in machine-assisted medical decision making. Nature Machine Intelligence 1, 1 (jan 2019), 20–23. https://doi.org/10.1038/s42256-018-0004-1

[9]

Sarah Belia, Fiona Fidler, Jennifer Williams, and Geoff Cumming. 2005. Researchers Misunderstand Confidence Intervals and Standard Error Bars.Psychological Methods 10, 4 (2005), 389–396. https://doi.org/10.1037/1082-989x.10.4.389

[10]

Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (Virtual Event, Canada) (FAccT ’21). Association for Computing Machinery, New York, NY, USA, 610–623. https://doi.org/10.1145/3442188.3445922

Digital Library

[11]

Jesse Josua Benjamin, Arne Berger, Nick Merrill, and James Pierce. 2021. Machine Learning Uncertainty as a Design Material: A Post-Phenomenological Inquiry. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 171, 14 pages. https://doi.org/10.1145/3411764.3445481

Digital Library

[12]

Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. Polity Press, Cambridge, UK.

[13]

Umang Bhatt, Javier Antorán, Yunfeng Zhang, Q. Vera Liao, Prasanna Sattigeri, Riccardo Fogliato, Gabrielle Melançon, Ranganath Krishnan, Jason Stanley, Omesh Tickoo, Lama Nachman, Rumi Chunara, Madhulika Srikumar, Adrian Weller, and Alice Xiang. 2021. Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (Virtual Event, USA) (AIES ’21). Association for Computing Machinery, New York, NY, USA, 401–413. https://doi.org/10.1145/3461702.3462571

Digital Library

[14]

Abeba Birhane. 2021. The Impossibility of Automating Ambiguity. Artificial Life 27, 1 (06 2021), 44–61. https://doi.org/10.1162/artl_a_00336 arXiv:https://direct.mit.edu/artl/article-pdf/27/1/44/2020407/artl_a_00336.pdf

[15]

Ryan David Bowler, Benjamin Bach, and Larissa Pschetz. 2022. Exploring Uncertainty in Digital Scheduling, and The Wider Implications of Unrepresented Temporalities in HCI. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 140, 12 pages. https://doi.org/10.1145/3491102.3502107

Digital Library

[16]

Gregor Broll and Steve Benford. 2005. Seamful Design for Location-Based Mobile Games. In Entertainment Computing - ICEC 2005, Fumio Kishino, Yoshifumi Kitamura, Hirokazu Kato, and Noriko Nagata (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 155–166.

Digital Library

[17]

Zana Buçinca, Phoebe Lin, Krzysztof Z. Gajos, and Elena L. Glassman. 2020. Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI ’20). Association for Computing Machinery, New York, NY, USA, 454–464. https://doi.org/10.1145/3377325.3377498

Digital Library

[18]

Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z. Gajos. 2021. To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-Assisted Decision-Making. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 188 (apr 2021), 21 pages. https://doi.org/10.1145/3449287

Digital Library

[19]

Adrian Bussone, Simone Stumpf, and Dympna O'Sullivan. 2015. The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems. In 2015 International Conference on Healthcare Informatics. IEEE. https://doi.org/10.1109/ichi.2015.26

Digital Library

[20]

Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. "Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 104 (nov 2019), 24 pages. https://doi.org/10.1145/3359206

Digital Library

[21]

Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2021. Onboarding Materials as Cross-Functional Boundary Objects for Developing AI Assistants. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, Article 43, 7 pages. https://doi.org/10.1145/3411763.3443435

Digital Library

[22]

Alexander Campolo and Kate Crawford. 2020. Enchanted Determinism: Power without Responsibility in Artificial Intelligence. Engaging Science, Technology, and Society 6 (January 2020), 1–19. https://www.microsoft.com/en-us/research/publication/enchanted-determinism-power-without-responsibility-in-artificial-intelligence/

[23]

Matthew Chalmers. 2003. Seamful Design and Ubicomp Infrastructure.

[24]

K Charmaz. 2003. Grounded theory: Objectivist and constructivist methods, in (Eds, Denzin, NK and Lincoln, YS) Grounded theory: Objectivist and constructivist methods, pp249-91.

[25]

Dean De Cock. 2011. Ames, Iowa: Alternative to the Boston Housing Data as an End of Semester Regression Project. Journal of Statistics Education 19, 3 (2011), null. https://doi.org/10.1080/10691898.2011.11889627 arXiv:https://doi.org/10.1080/10691898.2011.11889627

[26]

Michael Correll, Dominik Moritz, and Jeffrey Heer. 2018. Value-Suppressing Uncertainty Palettes. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3173574.3174216

Digital Library

[27]

Ray Crozier, Rob Ranyard, and Ola Svenson. 1997. Decision making : cognitive models and explanations (1st edition ed.). Routledge. 272 pages. http://site.ebrary.com/id/10057145

[28]

Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. (feb 2017). arxiv:1702.08608http://arxiv.org/abs/1702.08608

[29]

Upol Ehsan, Q. Vera Liao, Michael Muller, Mark O. Riedl, and Justin D. Weisz. 2021. Expanding Explainability: Towards Social Transparency in AI Systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 82, 19 pages. https://doi.org/10.1145/3411764.3445188

Digital Library

[30]

Upol Ehsan, Q. Vera Liao, Michael Muller, Mark O. Riedl, and Justin D. Weisz. 2021. Expanding Explainability: Towards Social Transparency in AI Systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 82, 19 pages. https://doi.org/10.1145/3411764.3445188

Digital Library

[31]

K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data.The MIT Press, Cambridge, MA, US. 426 pages.

[32]

Nel Escher and Nikola Banovic. 2020. Exposing Error in Poverty Management Technology: A Method for Auditing Government Benefits Screening Tools. Proc. ACM Hum.-Comput. Interact. 4, CSCW1, Article 64 (may 2020), 20 pages. https://doi.org/10.1145/3392874

Digital Library

[33]

Motahhare Eslami. 2017. Embracing Seamfulness and Uncertainty in Designing around Hidden Algorithms.

[34]

Kawin Ethayarajh. 2020. Is Your Classifier Actually Biased? Measuring Fairness under Uncertainty with Bernstein Bounds. (Apr 2020). arxiv:2004.12332http://arxiv.org/abs/2004.12332

[35]

Virginia Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.

Digital Library

[36]

Shi Feng and Jordan Boyd-Graber. 2019. What Can AI Do for Me? Evaluating Machine Learning Interpretations in Cooperative Play. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 229–239. https://doi.org/10.1145/3301275.3302265

Digital Library

[37]

Michael Fernandes, Logan Walls, Sean Munson, Jessica Hullman, and Matthew Ka. 2018. Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making. (2018). https://doi.org/10.1145/3173574.3173718

Digital Library

[38]

Milton Friedman and L. J. Savage. 1948. The Utility Analysis of Choices Involving Risk. Journal of Political Economy 56 (1948). https://EconPapers.repec.org/RePEc:ucp:jpolec:v:56:y:1948:p:279

[39]

Michael Froehlich, Charlotte Kobiella, Albrecht Schmidt, and Florian Alt. 2021. Is It Better With Onboarding? Improving First-Time Cryptocurrency App Experiences. In Designing Interactive Systems Conference 2021 (Virtual Event, USA) (DIS ’21). Association for Computing Machinery, New York, NY, USA, 78–89. https://doi.org/10.1145/3461778.3462047

Digital Library

[40]

Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In 33rd International Conference on Machine Learning, ICML 2016, Vol. 3. 1651–1660. arxiv:1506.02142

[41]

Soumya Ghosh, Q. Vera Liao, Karthikeyan Natesan Ramamurthy, Jiri Navratil, Prasanna Sattigeri, Kush Varshney, and Yunfeng Zhang. 2022. Uncertainty Quantification 360: A Hands-on Tutorial. In 5th Joint International Conference on Data Science &; Management of Data (9th ACM IKDD CODS and 27th COMAD) (Bangalore, India) (CODS-COMAD 2022). Association for Computing Machinery, New York, NY, USA, 333–335. https://doi.org/10.1145/3493700.3493767

Digital Library

[42]

Lisa Gitelman. 2013. Raw data is an oxymoron. MIT press.

[43]

Cleotilde Gonzalez. 2017. Decision-making: A cognitive science perspective. In The Oxford handbook of cognitive science.Oxford University Press, New York, NY, US, 249–263.

[44]

Germán González Rodríguez, Jose M Gonzalez-Cava, and Juan Albino Méndez Pérez. 2020. An intelligent decision support system for production planning based on machine learning. Journal of Intelligent Manufacturing 31, 5 (2020), 1257–1273. https://doi.org/10.1007/s10845-019-01510-y

Digital Library

[45]

Mitchell L. Gordon, Kaitlyn Zhou, Kayur Patel, Tatsunori Hashimoto, and Michael S. Bernstein. 2021. The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 388, 14 pages. https://doi.org/10.1145/3411764.3445423

Digital Library

[46]

George Anthony Gorry and Michael S. Scott Morton. 1971. A framework for management information systems. Working Paper.

[47]

Ben Green. 2022. The flaws of policies requiring human oversight of government algorithms. Computer Law & Security Review 45 (2022), 105681. https://doi.org/10.1016/j.clsr.2022.105681

[48]

Ben Green and Yiling Chen. 2019. The Principles and Limits of Algorithm-in-the-Loop Decision Making. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 50 (nov 2019), 24 pages. https://doi.org/10.1145/3359152

Digital Library

[49]

Miriam Greis, Passant El. Agroudy, Hendrik Schuff, Tonja Machulla, and Albrecht Schmidt. 2016. Decision-Making under Uncertainty: How the Amount of Presented Uncertainty Influences User Behavior. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction (Gothenburg, Sweden) (NordiCHI ’16). Association for Computing Machinery, New York, NY, USA, Article 52, 4 pages. https://doi.org/10.1145/2971485.2971535

Digital Library

[50]

Miriam Greis, Jessica Hullman, Michael Correll, Matthew Kay, and Orit Shaer. 2017. Designing for Uncertainty in HCI: When Does Uncertainty Help?. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA ’17). Association for Computing Machinery, New York, NY, USA, 593–600. https://doi.org/10.1145/3027063.3027091

Digital Library

[51]

Shunan Guo, Fan Du, Sana Malik, Eunyee Koh, Sungchul Kim, Zhicheng Liu, Donghyun Kim, Hongyuan Zha, and Nan Cao. 2019. Visualizing Uncertainty and Alternatives in Event Sequence Predictions. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300803

Digital Library

[52]

Nicole Hengesbach. 2022. Undoing Seamlessness: Exploring Seams for Critical Visualization. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 364, 7 pages. https://doi.org/10.1145/3491101.3519703

Digital Library

[53]

Morten Hertzum, Kristin D Hansen, and Hans H K Andersen. 2009. Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload?Behaviour & Information Technology 28, 2 (2009), 165–181. https://doi.org/10.1080/01449290701773842

Digital Library

[54]

M Hino, E Benami, and N Brooks. 2018. Machine learning for environmental monitoring. Nature Sustainability 1, 10 (2018), 583–588. https://doi.org/10.1038/s41893-018-0142-9

[55]

Jessica Hullman. [n. d.]. Why Authors Don’t Visualize Uncertainty. ([n. d.]).

[56]

Jessica Hullman. 2020. Why Authors Don’t Visualize Uncertainty. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 130–139. https://doi.org/10.1109/TVCG.2019.2934287

[57]

Jessica Hullman, Matthew Kay, Yea-Seul Kim, and Samana Shrestha. [n. d.]. Imagining Replications: Graphical Prediction & Discrete Visualizations Improve Recall & Estimation of Effect Uncertainty. ([n. d.]).

[58]

Jessica Hullman, Paul Resnick, and Eytan Adar. 2015. Hypothetical Outcome Plots Outperform Error Bars and Violin Plots for Inferences About Reliability of Variable Ordering. PLOS ONE 10, 11 (2015). http://idl.cs.washington.edu/papers/hops

[59]

Sarah Inman and David Ribes. 2019. "Beautiful Seams": Strategic Revelations and Concealments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300508

Digital Library

[60]

Philip N. Johnson-Laird. 1983. Mental models : towards a cognitive science of language, inference, and consciousness. Cambridge, MA: Harvard University Press. 528 p. pages. https://hal.archives-ouvertes.fr/hal-00702919Excerpts available on Google Books.

[61]

Ekaterina Jussupow, Kai Spohrer, Armin Heinzl, and Joshua Gawlitza. 2021. Augmenting Medical Diagnosis Decisions? An Investigation into Physicians’ Decision-Making Process with Artificial Intelligence. Information Systems Research 32, 3 (Sept. 2021), 713–735. https://doi.org/10.1287/isre.2020.0980

Digital Library

[62]

Daniel Kahneman and Amos Tversky. 1988. Prospect theory: An analysis of decision under risk.Cambridge University Press, New York, NY, US. 183–214 pages. https://doi.org/10.1017/CBO9780511609220.014

[63]

Alex Kale, Matthew Kay, and Jessica Hullman. 2021. Visual Reasoning Strategies for Effect Size Judgments and Decisions. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2021), 272–282. https://doi.org/10.1109/TVCG.2020.3030335

[64]

Harmanpreet Kaur, Eytan Adar, Eric Gilbert, and Cliff Lampe. 2022. Sensible AI: Re-Imagining Interpretability and Explainability Using Sensemaking Theory. In 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT ’22). Association for Computing Machinery, New York, NY, USA, 702–714. https://doi.org/10.1145/3531146.3533135

Digital Library

[65]

Anna Kawakami, Venkatesh Sivaraman, Logan Stapleton, Hao-Fei Cheng, Adam Perer, Zhiwei Steven Wu, Haiyi Zhu, and Kenneth Holstein. 2022. “Why Do I Care What’s Similar?” Probing Challenges in AI-Assisted Child Welfare Decision-Making through Worker-AI Interface Design Concepts. In Designing Interactive Systems Conference (Virtual Event, Australia) (DIS ’22). Association for Computing Machinery, New York, NY, USA, 454–470. https://doi.org/10.1145/3532106.3533556

Digital Library

[66]

Matthew Kay, Tara Kola, Jessica R. Hullman, and Sean A. Munson. 2016. When (ish) is my bus? User-centered visualizations of uncertainty in everyday, mobile predictive systems. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/2858036.2858558

Digital Library

[67]

Matthew Kay, Gregory L. Nelson, and Eric B. Hekler. 2016. Researcher-Centered Design of Statistics: Why Bayesian Statistics Better Fit the Culture and Incentives of HCI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 4521–4532. https://doi.org/10.1145/2858036.2858465

Digital Library

[68]

Ralph L. Keeney, Howard Raiffa, and David W. Rajala. 1979. Decisions with Multiple Objectives: Preferences and Value Trade-Offs. IEEE Transactions on Systems, Man, and Cybernetics 9 (1979), 403–403.

[69]

Amirhossein Kiani, Bora Uyumazturk, Pranav Rajpurkar, Alex Wang, Rebecca Gao, Erik Jones, Yifan Yu, Curtis P Langlotz, Robyn L Ball, Thomas J Montine, Brock A Martin, Gerald J Berry, Michael G Ozawa, Florette K Hazard, Ryanne A Brown, Simon B Chen, Mona Wood, Libby S Allard, Lourdes Ylagan, Andrew Y Ng, and Jeanne Shen. 2020. Impact of a deep learning assistant on the histopathologic classification of liver cancer. npj Digital Medicine 3, 1 (2020), 23. https://doi.org/10.1038/s41746-020-0232-8

[70]

Rafal Kocielnik, Saleema Amershi, and Paul N. Bennett. 2019. Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-User Expectations of AI Systems. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300641

Digital Library

[71]

Vivian Lai, Chacha Chen, Q. Vera Liao, Alison Smith-Renner, and Chenhao Tan. 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. https://doi.org/10.48550/ARXIV.2112.11471

[72]

Kathryn Ann Lambe, Gary O’Reilly, Brendan D Kelly, and Sarah Curristan. 2016. Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ Qual. Saf. 25, 10 (Oct. 2016), 808–820.

[73]

Zhiyuan Jerry Lin, Jongbin Jung, Sharad Goel, and Jennifer Skeem. 2020. The limits of human predictions of recidivism. Science Advances 6, 7 (2020). https://doi.org/10.1126/SCIADV.AAZ0652/SUPPL_FILE/AAZ0652_SM.PDF

[74]

Zachary C. Lipton. 2018. The Mythos of Model Interpretability: In Machine Learning, the Concept of Interpretability is Both Important and Slippery.Queue 16, 3 (jun 2018), 31–57. https://doi.org/10.1145/3236386.3241340

Digital Library

[75]

David J. C. MacKay. 1992. A Practical Bayesian Framework for Backpropagation Networks. Neural Computation 4, 3 (1992), 448–472. https://doi.org/10.1162/neco.1992.4.3.448

Digital Library

[76]

Charles F. Manski. 2020. The lure of incredible certitude. Economics and Philosophy 36, 2 (2020), 216–245. https://doi.org/10.1017/S0266267119000105

[77]

Sean McGrath, Parth Mehta, Alexandra Zytek, Isaac Lage, and Himabindu Lakkaraju. 2020. When does uncertainty matter?: Understanding the impact of predictive uncertainty in ML assisted decision making. arXiv (2020). arxiv:2011.06167

[78]

Sally Engle Merry. [n. d.]. The Seductions of Quantification: Measuring Human Rights, Gender Violence, and Sex Trafficking. Vol. 100. Chicago: University of Chicago Press.

[79]

Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (Atlanta, GA, USA) (FAT* ’19). Association for Computing Machinery, New York, NY, USA, 220–229. https://doi.org/10.1145/3287560.3287596

Digital Library

[80]

Ramaravind K. Mothilal, Amit Sharma, and Chenhao Tan. 2020. Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* ’20). Association for Computing Machinery, New York, NY, USA, 607–617. https://doi.org/10.1145/3351095.3372850

Digital Library

[81]

Safiya Umoja Noble. 2018. Algorithms of oppression. In Algorithms of Oppression. New York University Press.

[82]

Cathy O’neil. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway books.

[83]

Judith Orasanu and Terry Connolly. 1993. The reinvention of decision making. In Decision making in action: Models and methods.Ablex Publishing, Westport, CT, US, 3–20.

[84]

Daniel J Power. 2008. Decision Support Systems: A Historical Overview BT - Handbook on Decision Support Systems 1: Basic Themes. Springer Berlin Heidelberg, Berlin, Heidelberg, 121–140. https://doi.org/10.1007/978-3-540-48713-5_7

[85]

Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. 2022. Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI. https://doi.org/10.48550/ARXIV.2204.01075

[86]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Model-Agnostic Interpretability of Machine Learning. https://doi.org/10.48550/ARXIV.1606.05386

[87]

Téo Sanchez, Baptiste Caramiaux, Pierre Thiel, and Wendy E. Mackay. 2022. Deep Learning Uncertainty in Machine Teaching. In 27th International Conference on Intelligent User Interfaces (Helsinki, Finland) (IUI ’22). Association for Computing Machinery, New York, NY, USA, 173–190. https://doi.org/10.1145/3490099.3511117

Digital Library

[88]

Hugo Scurto, Baptiste Caramiaux, and Frederic Bevilacqua. 2021. Prototyping Machine Learning Through Diffractive Art Practice. In Designing Interactive Systems Conference 2021 (Virtual Event, USA) (DIS ’21). Association for Computing Machinery, New York, NY, USA, 2013–2025. https://doi.org/10.1145/3461778.3462163

Digital Library

[89]

Leonard M Silva, Emilio M Pereira, Paulo GO Salles, Ran Godrich, Rodrigo Ceballos, Jeremy D Kunz, Adam Casson, Julian Viret, Sarat Chandarlapaty, Carlos Gil Ferreira, Bruno Ferrari, Brandon Rothrock, Patricia Raciti, Victor Reuter, Belma Dogdas, George DeMuth, Jillian Sue, Christopher Kanan, Leo Grady, Thomas J Fuchs, and Jorge S Reis-Filho. 2021. Independent real-world application of a clinical-grade automated prostate cancer detection system. The Journal of Pathology 254, 2 (April 2021), 147–158. https://doi.org/10.1002/path.5662

[90]

Herbert A. Simon. 1956. Rational choice and the structure of the environment.Psychological Review 63 (1956), 129–138. https://doi.org/10.1037/h0042769

[91]

Meredith Skeels, Bongshin Lee, Greg Smith, and George Robertson. 2008. Revealing Uncertainty for Information Visualization. In Proceedings of the Working Conference on Advanced Visual Interfaces (Napoli, Italy) (AVI ’08). Association for Computing Machinery, New York, NY, USA, 376–379. https://doi.org/10.1145/1385569.1385637

Digital Library

[92]

Robert Soden, Lydia Chilton, Scott Miles, Rebecca Bicksler, Kaira Ray Villanueva, and Melissa Bica. 2022. Insights and Opportunities for HCI Research into Hurricane Risk Communication. In CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 325, 13 pages. https://doi.org/10.1145/3491102.3502101

Digital Library

[93]

Robert Soden, Laura Devendorf, Richmond Wong, Yoko Akama, and Ann Light. 2022. Modes of Uncertainty in HCI. Foundations and Trends® in Human–Computer Interaction 15, 4 (2022), 317–426. https://doi.org/10.1561/1100000085

[94]

Robert Soden, Laura Devendorf, Richmond Y. Wong, Lydia B. Chilton, Ann Light, and Yoko Akama. 2020. Embracing Uncertainty in HCI. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3375177

Digital Library

[95]

Katta Spiel, Oliver L. Haimson, and Danielle Lottridge. 2019. How to Do Better with Gender on Surveys: A Guide for HCI Researchers. Interactions 26, 4 (jun 2019), 62–65. https://doi.org/10.1145/3338283

Digital Library

[96]

Alvin T. Tan. 2019. Cracking the Ames Housing Dataset with Linear Regression. https://github.com/at-tan/Cracking_Ames_Housing_OLS.

[97]

Michael E. Tipping. 2001. Sparse Bayesian Learning and the Relevance Vector Machine. J. Mach. Learn. Res. 1 (sep 2001), 211–244. https://doi.org/10.1162/15324430152748236

Digital Library

[98]

Richard Tomsett, Alun Preece, Dave Braines, Federico Cerutti, Supriyo Chakraborty, Mani Srivastava, Gavin Pearson, and Lance Kaplan. 2020. Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI. Patterns 1, 4 (2020), 100049. https://doi.org/10.1016/j.patter.2020.100049

[99]

Sana Tonekaboni, Shalmali Joshi, Melissa D. McCradden, and Anna Goldenberg. 2019. What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use. 106 (09–10 Aug 2019), 359–380. https://proceedings.mlr.press/v106/tonekaboni19a.html

[100]

Amos Tversky and Daniel Kahneman. 1974. Judgment under Uncertainty: Heuristics and Biases. Science 185, 4157 (1974), 1124–1131. https://doi.org/10.1126/science.185.4157.1124

[101]

Laurent Valentin Jospin, Wray Buntine, Farid Boussaid, Hamid Laga, and Mohammed Bennamoun. 2020. Hands-on Bayesian Neural Networks - a Tutorial for Deep Learning Users. arxiv:2007.06823

[102]

Hilde J. P. Weerts, Werner van Ipenburg, and Mykola Pechenizkiy. 2019. A Human-Grounded Evaluation of SHAP for Alert Processing. https://doi.org/10.48550/ARXIV.1907.03324

[103]

Karl E Weick. 1995. Sensemaking in organizations. Vol. 3. Sage.

[104]

Mark Weiser. 1999. The Computer for the 21st Century. SIGMOBILE Mob. Comput. Commun. Rev. 3, 3 (jul 1999), 3–11. https://doi.org/10.1145/329124.329126

Digital Library

[105]

John H. Williamson, Antti Oulasvirta, Per Ola Kristensson, and Nikola Banovic. 2022. Bayesian Methods for Interaction and Design. Cambridge University Press.

[106]

Haiping Xu and Amol Gade. 2017. Smart real estate assessments using structured deep neural networks. In 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). 1–7. https://doi.org/10.1109/UIC-ATC.2017.8397560

[107]

Jiayi Xu. 2021. A Novel Deep Neural Network based Method for House Price Prediction. In 2021 International Conference of Social Computing and Digital Economy (ICSCDE). 12–16. https://doi.org/10.1109/ICSCDE54196.2021.00012

[108]

Yunfeng Zhang, Q. Vera Liao, and Rachel K. E. Bellamy. 2020. Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* ’20). Association for Computing Machinery, New York, NY, USA, 295–305. https://doi.org/10.1145/3351095.3372852

Digital Library

Cited By

Cohen JHuan X(2024)Uncertainty-aware explainable AI as a foundational paradigm for digital twinsFrontiers in Mechanical Engineering10.3389/fmech.2023.13291469Online publication date: 5-Jan-2024
https://doi.org/10.3389/fmech.2023.1329146
Kaur HConrad MRule DLampe CGilbert E(2024)Interpretability Gone Bad: The Role of Bounded Rationality in How Practitioners Understand Machine LearningProceedings of the ACM on Human-Computer Interaction10.1145/36373548:CSCW1(1-34)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637354
Kim SLiao QVorvoreanu MBallard SVaughan J(2024)"I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and TrustProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658941(822-835)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658941
Show More Cited By

Index Terms

Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI theory, concepts and models
  2. Visualization
    1. Empirical studies in visualization
2. Information systems
  1. Information systems applications
    1. Decision support systems

Recommendations

Making Better Decision Makers

Decision analysis has been used to help solve numerous complex decisions over the last few decades. However, its power as a basis for structuring one's thinking to resolve decisions has barely been tapped. To realize this potential, we in the decision ...
Decision Making with Imperfect Decision Makers
On multicriteria decision making under conditions of uncertainty

This paper deals with multicriteria decision making problems under conditions of uncertainty. The main its contribution is the consideration of choice criteria of the classic approach to handle information uncertainty in monocriteria decision making as ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '23: Proceedings of the 28th International Conference on Intelligent User Interfaces

March 2023

972 pages

ISBN:9798400701061

DOI:10.1145/3581641

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 March 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IUI '23

Sponsor:

IUI '23: 28th International Conference on Intelligent User Interfaces

March 27 - 31, 2023

NSW, Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
1,341
Total Downloads

Downloads (Last 12 months)885
Downloads (Last 6 weeks)206

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cohen JHuan X(2024)Uncertainty-aware explainable AI as a foundational paradigm for digital twinsFrontiers in Mechanical Engineering10.3389/fmech.2023.13291469Online publication date: 5-Jan-2024
https://doi.org/10.3389/fmech.2023.1329146
Kaur HConrad MRule DLampe CGilbert E(2024)Interpretability Gone Bad: The Role of Bounded Rationality in How Practitioners Understand Machine LearningProceedings of the ACM on Human-Computer Interaction10.1145/36373548:CSCW1(1-34)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637354
Kim SLiao QVorvoreanu MBallard SVaughan J(2024)"I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and TrustProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658941(822-835)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658941
Wu CChakravorti TCarroll JRajtmajer S(2024)Integrating measures of replicability into scholarly search: Challenges and opportunitiesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3643043(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3643043
Schömbs SPareek SGoncalves JJohal W(2024)Robot-Assisted Decision-Making: Unveiling the Role of Uncertainty Visualisation and EmbodimentProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642911(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642911
Zhang DChatzimparmpas AKamali NHullman J(2024)Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image LabelingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642446(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642446
Jin ASalehi N(2024)(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in CourtProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641902(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641902
Saheb TSaheb T(2024)Mapping Ethical Artificial Intelligence Policy Landscape: A Mixed Method AnalysisScience and Engineering Ethics10.1007/s11948-024-00472-630:2Online publication date: 7-Mar-2024
https://doi.org/10.1007/s11948-024-00472-6
van Gerwen SConstantino JRoothaert RWeerheijm BWagner BPavlin GKlievink BSchlobach STuma KMassacci F(2024)To Know What You Do Not Know: Challenges for Explainable AI for Security and Threat IntelligenceArtificial Intelligence for Security10.1007/978-3-031-57452-8_4(55-83)Online publication date: 17-Apr-2024
https://doi.org/10.1007/978-3-031-57452-8_4
Radensky MSéguin JLim JOlson KGeiger R(2023)“I Think You Might Like This”: Exploring Effects of Confidence Signal Patterns on Trust in and Reliance on Conversational Recommender SystemsProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594043(792-804)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594043
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents