research-article

Guidelines for Human-AI Interaction

Authors:

Saleema Amershi,

Mihaela Vorvoreanu,

Penny Collisson,

Paul N. Bennett,

Ruth Kikin-Gil,

Eric HorvitzAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 3, Pages 1 - 13

https://doi.org/10.1145/3290605.3300233

Published: 02 May 2019 Publication History

Abstract

Advances in artificial intelligence (AI) frame opportunities and challenges for user interface design. Principles for human-AI interaction have been discussed in the human-computer interaction community for over two decades, but more study and innovation are needed in light of advances in AI and the growing uses of AI technologies in human-facing applications. We propose 18 generally applicable design guidelines for human-AI interaction. These guidelines are validated through multiple rounds of evaluation including a user study with 49 design practitioners who tested the guidelines against 20 popular AI-infused products. The results verify the relevance of the guidelines over a spectrum of interaction scenarios and reveal gaps in our knowledge, highlighting opportunities for further research. Based on the evaluations, we believe the set of design guidelines can serve as a resource to practitioners working on the design of applications and features that harness AI technologies, and to researchers interested in the further development of human-AI interaction design principles.

Supplementary Material

ZIP File (paper003.zip)

The auxiliary material consists of a single zip file containing a PDF of the paper's appendix. In this appendix, we illustrate each of our 18 human-AI interaction design guidelines with three example applications and three example violations provided by our user study participants when testing the guidelines against popular AI-infused products. For each example we indicate the product category the participant was testing, but obscure the specific product names.

Download
213.57 KB

References

[1]

Alexa. 2018. Top sites in the United States. Retrieved July, 2018 from https://www.alexa.com/topsites/countries/US

[2]

Kathy Baxter. 2017. How to Meet User Expectations for Artificial Intelligence. Medium. Retrieved September, 2018 from https://medium.com/salesforce-ux/how-to-meet-userexpectations-for-artificial-intelligence-a51d3c82af6

[3]

Victoria Bellotti and Keith Edwards. 2001. Intelligibility and Accountability: Human Considerations in Context-Aware Systems. Human-- Computer Interaction 16, 2--4 (2001), 193--212.

Digital Library

[4]

Andrea Bunt, Matthew Lount, and Catherine Lauzon. 2012. Are Explanations Always Important?: A Study of Deployed, Low-cost Intelligent Interactive Systems. In Proc. IUI '12. ACM, New York, NY, USA, 169-- 178.

Digital Library

[5]

Margaret Burnett, Simone Stumpf, Jamie Macbeth, Stephann Makri, Laura Beckwith, Irwin Kwan, Anicia Peters, and William Jernigan. 2016. GenderMag: A method for evaluating software's gender inclusiveness. Interacting with Computers 28, 6 (2016), 760--787.

[6]

comScore. 2018. Latest rankings. Retrieved July, 2018 from https: //www.comscore.com/Insights/Rankings

[7]

Maartje de Graaf, Somaya Ben Allouch, and Jan van Dijk. 2017. Why Do They Refuse to Use My Robot?: Reasons for Non-Use Derived from a Long-Term Home Study. In Proc. HRI '17. ACM, New York, NY, USA, 224--233.

Digital Library

[8]

Defy Media. 2015. Damn You Auto Correct! Retrieved September, 2018 from http://www.damnyouautocorrect.com/

[9]

T. Deuschel and T. Scully. 2016. On the Importance of Spatial Perception for the Design of Adaptive User Interfaces. In 2016 IEEE 10th International Conference on Self-Adaptive and Self-Organizing Systems (SASO). 70--79.

[10]

Anind Dey, Jennifer Mankoff, Gregory Abowd, and Scott Carter. 2002. Distributed Mediation of Ambiguous Context in Aware Environments. In Proc. UIST '02. ACM, New York, NY, USA, 121--130.

Digital Library

[11]

Leah Findlater and Joanna McGrenere. 2004. A Comparison of Static, Adaptive, and Adaptable Menus. In Proc. CHI '04. ACM, New York, NY, USA, 89--96.

Digital Library

[12]

Krzysztof Z. Gajos, Mary Czerwinski, Desney S. Tan, and Daniel S. Weld. 2006. Exploring the Design Space for Adaptive Graphical User Interfaces. In Proc. AVI '06. ACM, New York, NY, USA, 201--208.

Digital Library

[13]

Krzysztof Z Gajos, Katherine Everitt, Desney S Tan, Mary Czerwinski, and Daniel S Weld. 2008. Predictability and accuracy in adaptive user interfaces. In CHI. ACM, 1271--1274.

Digital Library

[14]

Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining Collaborative Filtering Recommendations. In Proc. CSCW '00. ACM, New York, NY, USA, 241--250.

Digital Library

[15]

Setia Hermawati and Glyn Lawson. 2016. Establishing usability heuristics for heuristics evaluation in a specifc domain: Is there a consensus? Applied Ergonomics 56 (2016), 34 -- 51.

[16]

Kristina Höök. 2000. Steps to take before intelligent user interfaces become real. Interacting with Computers 12, 4 (2000), 409--426.

[17]

Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. In Proc. CHI '99. ACM, New York, NY, USA, 159--166.

Digital Library

[18]

Eric Horvitz, Jack Breese, David Heckerman, David Hovel, and Koos Rommelse. 1998. The Lumière Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. In Proc. UAI '98. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 256--265.

Digital Library

[19]

IDC Corporate USA. 2018. IDC: The premier global market intelligence firm. Retrieved September, 2018 from https://www.idc.com/

[20]

Anthony Jameson. 2008. Adaptive interfaces and agents. In The human-computer interaction handbook: Fundamentals, evolving technologies and emerging applications (2nd ed.), Andrew Sears and Julie A. Jacko (Eds.). CRC Press, Boca Raton, FL, 433--458.

Digital Library

[21]

Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of Explanatory Debugging to Personalize Interactive Machine Learning. In Proc. IUI '15. ACM, New York, NY, USA, 126--137.

Digital Library

[22]

Chia-Jung Lee, Jaime Teevan, and Sebastian de la Chica. 2014. Characterizing Multi-click Search Behavior and the Risks and Opportunities of Changing Results During Use. In Proc. SIGIR '14. ACM, New York, NY, USA, 515--524.

Digital Library

[23]

Brian Y. Lim and Anind K. Dey. 2009. Assessing Demand for Intelligibility in Context-aware Applications. In Proc. UbiComp '09. ACM, New York, NY, USA, 195--204.

Digital Library

[24]

Josh Lovejoy. 2018. The UX of AI. Google Design. Retrieved September, 2018 from https://design.google/library/ux-ai/

[25]

Josh Lovejoy and Jess Holbrook. 2017. Human-Centered Machine Learning. 7 steps to stay focused on the user when designing with ML. Medium. Retrieved September, 2018 from https://medium.com/google-design/human-centered-machinelearning-a770d10562cd

[26]

Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf Between User Expectation and Experience of Conversational Agents. In Proc. CHI '16. ACM, New York, NY, USA, 5286--5297.

Digital Library

[27]

Aaron Maitland and Stanley Presser. 2016. How Accurately Do Different Evaluation Methods Predict the Reliability of Survey Questions? Journal of Survey Statistics and Methodology 4, 3 (2016), 362--381.

[28]

Joseph A Maxwell. 2012. Qualitative research design: An interactive approach. Vol. 41. Sage publications.

[29]

Chelsea Myers, Anushay Furqan, Jessica Nebolsky, Karina Caro, and Jichen Zhu. 2018. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proc. CHI '18. ACM, New York, NY, USA, Article 6, 7 pages.

Digital Library

[30]

Jakob Nielsen. 1992. Finding Usability Problems Through Heuristic Evaluation. In Proc. CHI '92. ACM, New York, NY, USA, 373--380.

Digital Library

[31]

Jakob Nielsen and Rolf Molich. 1990. Heuristic Evaluation of User Interfaces. In Proc. CHI '90. ACM, New York, NY, USA, 249--256.

Digital Library

[32]

D.A. Norman. 1988. The psychology of everyday things. Basic Books, New York.

[33]

Donald A. Norman. 1994. How Might People Interact with Agents. Commun. ACM 37, 7 (July 1994), 68--71.

Digital Library

[34]

Kristen Olson. 2010. An examination of questionnaire evaluation by expert reviewers. Field Methods 22, 4 (2010), 295--318.

[35]

Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice Interfaces in Everyday Life. In Proc. CHI '18. ACM, New York, NY, USA, Article 640, 12 pages.

Digital Library

[36]

Emilee Rader, Kelley Cotter, and Janghee Cho. 2018. Explanations As Mechanisms for Supporting Algorithmic Transparency. In Proc. CHI '18. ACM, New York, NY, USA, Article 103, 13 pages.

Digital Library

[37]

Ruth Ravichandran, Sang-Wha Sien, Shwetak N. Patel, Julie A. Kientz, and Laura R. Pina. 2017. Making Sense of Sleep Sensors: How Sleep Sensing Technologies Support and Undermine Sleep Health. In Proc. CHI '17. ACM, New York, NY, USA, 6864--6875.

Digital Library

[38]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proc. KDD '16. ACM, New York, NY, USA, 1135--1144.

Digital Library

[39]

Katharine Schwab. 2017. 10 Principles For Design In The Age Of AI. Retrieved September, 2018 from https://www.fastcompany.com/3067632/10-principles-for-designin-the-age-of-ai

[40]

SimilarWeb. 2018. Top websites ranking. Retrieved July, 2018 from https://www.similarweb.com/top-websites/united-states

[41]

Jack Stewart. 2018. Why Tesla's Autopilot Can't See a Stopped Firetruck. Retrieved September, 2018 from https: //www.wired.com/story/tesla-autopilot-why-crash-radar/

[42]

Erica Virtue. 2017. Designing with AI.

[43]

Jolita Vveinhardt and Evelina Gulbovaite. 2016. Expert evaluation of diagnostic instrument for personal and organizational value congruence. Journal of business ethics 136, 3 (2016), 481--501.

[44]

Daniel S Weld and Gagan Bansal. 2018. Intelligible Artificial Intelligence. arXiv preprint arXiv:1803.04263 (2018).

[45]

Rayoung Yang, Eunice Shin, Mark W. Newman, and Mark S. Ackerman. 2015. When Fitness Trackers Don'T 'Fit': End-user Difficulties in the Assessment of Personal Tracking Device Accuracy. In Proc. UbiComp '15. ACM, New York, NY, USA, 623--634.

Digital Library

Cited By

Wang XWu Y(2024)Balancing Innovation and Regulation in the Age of Generative Artificial IntelligenceJournal of Information Policy10.5325/jinfopoli.14.2024.001214Online publication date: 2-Jul-2024
https://doi.org/10.5325/jinfopoli.14.2024.0012
Güzelci OKaradag I(2024)Revisiting the Key Components of Creativity Through Generative AIMaking Art With Generative AI Tools10.4018/979-8-3693-1950-5.ch001(1-16)Online publication date: 19-Apr-2024
https://doi.org/10.4018/979-8-3693-1950-5.ch001
B. Sable AKapse D(2024)A Framework Design for Centralised Monitoring of Patient Disease Diagnosis for Better ImprovementInternational Journal of Engineering and Advanced Technology10.35940/ijeat.D4438.1304042413:4(47-52)Online publication date: 30-Apr-2024
https://doi.org/10.35940/ijeat.D4438.13040424
Show More Cited By

Index Terms

Guidelines for Human-AI Interaction
1. Computing methodologies
  1. Artificial intelligence
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Assessing Human-AI Interaction Early through Factorial Surveys: A Study on the Guidelines for Human-AI Interaction
This work contributes a research protocol for evaluating human-AI interaction in the context of specific AI products. The research protocol enables UX and HCI researchers to assess different human-AI interaction solutions and validate design decisions ...
Human-AI Interaction and AI Avatars
HCI International 2023 – Late Breaking Papers
Abstract
Human-Computer Interaction has been evolving rapidly with the advancement of artificial intelligence and metaverse. Human-AI Interaction is a new area in Human-Computer Interaction. In this paper, we look at AI avatars, which are human-like ...
Designing for Continuous Interaction with Artificial Intelligence Systems
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

The increasing capabilities of Artificial Intelligence enable the support of users in a continuously growing number of applications. Current systems typically dictate that interaction between user input and AI output unfolds in discrete steps, as is ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

852
Total Citations
View Citations
23,086
Total Downloads

Downloads (Last 12 months)6,037
Downloads (Last 6 weeks)540

Reflects downloads up to 14 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang XWu Y(2024)Balancing Innovation and Regulation in the Age of Generative Artificial IntelligenceJournal of Information Policy10.5325/jinfopoli.14.2024.001214Online publication date: 2-Jul-2024
https://doi.org/10.5325/jinfopoli.14.2024.0012
Güzelci OKaradag I(2024)Revisiting the Key Components of Creativity Through Generative AIMaking Art With Generative AI Tools10.4018/979-8-3693-1950-5.ch001(1-16)Online publication date: 19-Apr-2024
https://doi.org/10.4018/979-8-3693-1950-5.ch001
B. Sable AKapse D(2024)A Framework Design for Centralised Monitoring of Patient Disease Diagnosis for Better ImprovementInternational Journal of Engineering and Advanced Technology10.35940/ijeat.D4438.1304042413:4(47-52)Online publication date: 30-Apr-2024
https://doi.org/10.35940/ijeat.D4438.13040424
Ghimire PKim KAcharya M(2024)Opportunities and Challenges of Generative AI in Construction Industry: Focusing on Adoption of Text-Based ModelsBuildings10.3390/buildings1401022014:1(220)Online publication date: 14-Jan-2024
https://doi.org/10.3390/buildings14010220
Di Sarno LCaroselli ATonin GGraglia BPansini VCausio FGatto AChiaretti A(2024)Artificial Intelligence in Pediatric Emergency Medicine: Applications, Challenges, and Future PerspectivesBiomedicines10.3390/biomedicines1206122012:6(1220)Online publication date: 30-May-2024
https://doi.org/10.3390/biomedicines12061220
Bauroth MRath-Manakidis PLangholf VWiskott LGlasmachers T(2024)tachAId—An interactive tool supporting the design of human-centered AI solutionsFrontiers in Artificial Intelligence10.3389/frai.2024.13541147Online publication date: 12-Mar-2024
https://doi.org/10.3389/frai.2024.1354114
Solis Pino ARuiz PAgredo-Delgado VMon ACollazos C(2024)Bibliometric Analysis of the Research Landscape in Human-Computer Interaction in Ibero-AmericaAnálisis bibliométrico del panorama de la investigación de la interacción humano-computadora en IberoaméricaTecnoLógicas10.22430/22565337.290727:59(e2907)Online publication date: 7-Mar-2024
https://doi.org/10.22430/22565337.2907
Depauw TBoasen JLéger PSénécal S(2024)Assessing the Relationship Between Digital Trail Making Test Performance and IT Task Performance: Empirical StudyJMIR Human Factors10.2196/4999211(e49992)Online publication date: 14-Jun-2024
https://doi.org/10.2196/49992
Lee SKoo Y(2024)Proposal of a Facilitation and Process Model for Enhancing Creativity in Co-design Workshops with Generative AI: The Use of ChatGPTArchives of Design Research10.15187/adr.2024.05.37.2.24937:2(249-281)Online publication date: 31-May-2024
https://doi.org/10.15187/adr.2024.05.37.2.249
Loof TSpinks RGagnon L(2024)My co-worker is a real tool: Perceptions of Human-AI teamworkNewhouse Impact Journal10.14305/jn.29960819.2024.1.1.051:1(25-30)Online publication date: 2024
https://doi.org/10.14305/jn.29960819.2024.1.1.05
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents