Embodied Machine Learning
Authors: 
Alexander Bakogeorge, Syeda Aniqa Imtiaz, Nour Abu Hantash, Roozbeh Manshaei, Ali MazalekAuthors Info & Claims
Article No.: 22, Pages 1 - 12
Abstract
Machine learning becomes more prevalent in specialized domains such as medicine and biology every year, but domain expert trust in machine learning continues to lag behind. Researchers have explored increasing rational trust in AI but little research exists focusing on systems that foster affective and normative trust between domain experts and data scientists who create the models. Tools like Project Jupyter have attempted to bridge this gap between data scientists and domain experts, but failed to see uptake in applied fields or to promote collaboration through co-located synchronous work. To address this we present a proof-of-concept tabletop interactive machine learning system for synchronous, co-located model fine tuning. We tested our system with biology experts and data scientists on a cell biology dataset. Results show that our system promotes interactions between domain experts, data scientists, and the model-in-training and fosters domain expert affective and normative trust in the resulting AI model.
Supplemental Material
MP4 File
Demo video
- Download
- 133.01 MB
References
[1]
2019. Ascent of machine learning in medicine. Nature Materials 18, 5 (01 May 2019), 407–407. https://doi.org/10.1038/s41563-019-0360-1
[2]
[2] [Accessed 2022]. https://jupyter.org/
[3]
[3] [Accessed 2022]. https://colab.research.google.com/notebooks
[4]
Ahmed Alqaraawi, Martin Schuessler, Philipp Weiß, Enrico Costanza, and Nadia Berthouze. 2020. Evaluating saliency map explanations for convolutional neural networks: a user study. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 275–285.
[5]
Christopher Andrews, Alex Endert, Beth Yost, and Chris North. 2011. Information visualization on large, high-resolution displays: Issues, challenges, and opportunities. Information Visualization 10, 4 (2011), 341–355.
[6]
Tita Alissa Bach, Amna Khan, Harry Hallock, Gabriela Beltrão, and Sonia Sousa. 2022. A systematic literature review of user trust in AI-enabled systems: An HCI perspective. International Journal of Human–Computer Interaction (2022), 1–16.
[7]
Patrick Bedué and Albrecht Fritzsche. 2021. Can we trust AI? An empirical investigation of trust requirements and guide to successful AI adoption. Journal of Enterprise Information Management (2021).
[8]
Arngeir Berge, Frode Guribye, Siri-Linn Schmidt Fotland, Gro Fonnes, Ingrid H Johansen, and Christoph Trattner. 2023. Designing for Control in Nurse-AI Collaboration During Emergency Medical Calls. In Proceedings of the 2023 ACM Designing Interactive Systems Conference. 1339–1352.
[9]
Charlotte Blease, Cosima Locher, Marisa Leon-Carlyle, and M Doraiswamy. 2020. Artificial intelligence and the future of psychiatry: qualitative findings from a global physician survey. Digital health 6 (2020), 2055207620968355.
[10]
Lauren Bradel, Alex Endert, Kristen Koch, Christopher Andrews, and Chris North. 2013. Large high resolution displays for co-located collaborative sensemaking: Display usage and territoriality. International Journal of Human-Computer Studies 71, 11 (2013), 1078–1088.
[11]
Francisco Maria Calisto, João Fernandes, Margarida Morais, Carlos Santiago, João Maria Abrantes, Nuno Nunes, and Jacinto C Nascimento. 2023. Assertiveness-based Agent Communication for a Personalized Medicine on Medical Imaging Diagnosis. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–20.
[12]
Francisco M Calisto, Alfredo Ferreira, Jacinto C Nascimento, and Daniel Gonçalves. 2017. Towards touch-based medical image diagnosis annotation. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces. 390–395.
[13]
Oliver Carr, Stojan Jovanovic, Luca Albergante, Fernando Andreotti, Robert Dürichen, Nadia Lipunova, Janie Baxter, Rabia Khan, and Benjamin Irving. 2020. Deep Semi-Supervised Embedded Clustering (DSEC) for Stratification of Heart Failure Patients. arXiv preprint arXiv:2012.13233 (2020).
[14]
Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 1721–1730.
[15]
Gromit Yeuk-Yin Chan, Enrico Bertini, Luis Gustavo Nonato, Brian Barr, and Claudio T Silva. 2020. Melody: generating and visualizing machine learning model summary to understand data and classifiers together. arXiv preprint arXiv:2007.10614 (2020).
[16]
Runpu Chen, Le Yang, Steve Goodison, and Yijun Sun. 2020. Deep-learning approach to identifying cancer subtypes using high-dimensional genomic data. Bioinformatics 36, 5 (2020), 1476–1483.
[17]
C Brad Crisp and Sirkka L Jarvenpaa. 2013. Swift trust in global virtual teams: Trusting beliefs and normative actions.Journal of personnel psychology 12, 1 (2013), 45.
[18]
Matthew F Dixon, Igor Halperin, and Paul Bilokon. 2020. Machine Learning in Finance. Springer.
[19]
Lavanya Elluri, Varun Mandalapu, and Nirmalya Roy. 2019. Developing machine learning based predictive models for smart policing. In 2019 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 198–204.
[20]
Niklas Elmqvist, Andrew Vande Moere, Hans-Christian Jetter, Daniel Cernea, Harald Reiterer, and TJ Jankun-Kelly. 2011. Fluid interaction for information visualization. Information Visualization 10, 4 (2011), 327–340.
[21]
Alex Endert. 2016. Semantic Interaction for Visual Analytics: Inferring Analytical Reasoning for Model Steering. Morgan & Claypool Publishers.
[22]
Evelyn Fix and Joseph Lawson Hodges. 1989. Discriminatory analysis. Nonparametric discrimination: Consistency properties. International Statistical Review/Revue Internationale de Statistique 57, 3 (1989), 238–247.
[23]
Karen Frost-Arnold. 2014. The cognitive attitude of rational trust. Synthese 191, 9 (2014), 1957–1974.
[24]
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9
[25]
Joshua James Hatherley. 2020. Limits of trust in medical AI. Journal of medical ethics 46, 7 (2020), 478–481.
[26]
Mikkel R Jakobsen and Kasper Hornbæk. 2014. Up close and personal: Collaborative work on a high-resolution multitouch wall display. ACM Transactions on Computer-Human Interaction (TOCHI) 21, 2 (2014), 1–34.
[27]
Magnus Høholt Kaspersen, Karl-Emil Kjær Bilstrup, and Marianne Graves Petersen. 2021. The Machine Learning Machine: A Tangible User Interface for Teaching Machine Learning. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction. 1–12.
[28]
Jack Kiefer and Jacob Wolfowitz. 1952. Stochastic estimation of the maximum of a regression function. The Annals of Mathematical Statistics (1952), 462–466.
[29]
Mi Jeong Kim and Mary Lou Maher. 2008. The impact of tangible user interfaces on spatial cognition during collaborative design. Design Studies 29, 3 (2008), 222–253.
[30]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[31]
Bran Knowles and John T Richards. 2021. The sanction of authority: Promoting public trust in ai. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 262–271.
[32]
Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics 22, 1 (1951), 79–86.
[33]
Minha Lee, Lily Frank, and Wijnand IJsselsteijn. 2021. Brokerbot: A cryptocurrency chatbot in the social-technical gap of trust. Computer Supported Cooperative Work (CSCW) 30, 1 (2021), 79–117.
[34]
Benjamin Letham, Cynthia Rudin, Tyler H McCormick, and David Madigan. 2015. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics 9, 3 (2015), 1350–1371.
[35]
Steven Lockey, Nicole Gillespie, Daniel Holm, and Ida Asadi Someh. 2021. A review of trust in artificial intelligence: Challenges, vulnerabilities and future directions. (2021).
[36]
Maria Madsen and Shirley Gregor. 2000. Measuring human-computer trust. In 11th australasian conference on information systems, Vol. 53. Citeseer, 6–8.
[37]
Roozbeh Manshaei, Nauman Baig, Sean Delong, Shahin Khayyer, Brien East, and Ali Mazalek. 2017. Tangible mtDNA: A Tangible Tabletop System for Exploring Genetic Mutations on Mitochondrial DNA Cancer Data. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction. 101–110.
[38]
Roozbeh Manshaei, Sean DeLong, Uzair Mayat, Dhrumil Patal, Matthew Kyan, and Ali Mazalek. 2019. Tangible bionets: Multi-surface and tangible interactions for exploring structural features of biological networks. Proceedings of the ACM on Human-Computer Interaction 3, EICS (2019), 1–22.
[39]
Yaoli Mao, Dakuo Wang, Michael Muller, Kush R Varshney, Ioana Baldini, Casey Dugan, and Aleksandra Mojsilović. 2019. How data scientists work together with domain experts in scientific collaborations: To find the right answer or to ask the right question?Proceedings of the ACM on Human-Computer Interaction 3, GROUP (2019), 1–23.
[40]
Magdalena Mateescu, Christoph Pimmer, Carmen Zahn, Daniel Klinkhammer, and Harald Reiterer. 2021. Collaboration on large interactive displays: a systematic review. Human–Computer Interaction 36, 3 (2021), 243–277.
[41]
Roger C Mayer, James H Davis, and F David Schoorman. 1995. An integrative model of organizational trust. Academy of management review 20, 3 (1995), 709–734.
[42]
Honghui Mei, Huihua Guan, Chengye Xin, Xiao Wen, and Wei Chen. 2020. Datav: Data visualization on large high-resolution displays. Visual Informatics 4, 3 (2020), 12–23.
[43]
T Nathan Mundhenk, Barry Y Chen, and Gerald Friedland. 2019. Efficient saliency maps for explainable AI. arXiv preprint arXiv:1911.11293 (2019).
[44]
Philip J Nickel. 2022. Trust in medical artificial intelligence: a discretionary account. Ethics and Information Technology 24, 1 (2022), 1–10.
[45]
Kazuo Okamura and Seiji Yamada. 2020. Adaptive trust calibration for human-AI collaboration. Plos one 15, 2 (2020), e0229132.
[46]
Claudia Perlich, Brian Dalessandro, Troy Raeder, Ori Stitelman, and Foster Provost. 2014. Machine learning for targeted display advertising: Transfer learning in action. Machine learning 95, 1 (2014), 103–127.
[47]
David Piorkowski, Soya Park, April Yi Wang, Dakuo Wang, Michael Muller, and Felix Portnoy. 2021. How ai developers overcome communication challenges in a multidisciplinary team: A case study. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–25.
[48]
Caroline Privault, Jacki O’Neill, Victor Ciriza, and Jean-Michel Renders. 2010. A new tangible user interface for machine learning document review. Artificial Intelligence and Law 18, 4 (2010), 459–479.
[49]
Khairi Reda, Andrew E Johnson, Michael E Papka, and Jason Leigh. 2015. Effects of display size and resolution on user behavior and insight acquisition in visual exploration. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2759–2768.
[50]
Mark Ryan. 2020. In AI we trust: ethics, artificial intelligence, and reliability. Science and Engineering Ethics 26, 5 (2020), 2749–2767.
[51]
Dominik Sacha, Michael Sedlmair, Leishi Zhang, John A Lee, Jaakko Peltonen, Daniel Weiskopf, Stephen C North, and Daniel A Keim. 2017. What you see is what you can change: Human-centered machine learning by interactive visualization. Neurocomputing 268 (2017), 164–175.
[52]
Jessica Zeitz Self, Leanna House, S Leman, and C North. 2015. Andromeda: Observation-level and parametric interaction for exploratory data analysis. Technical Report. Technical report, Department of Computer Science, Virginia Tech, Blacksburg ….
[53]
Orit Shaer, Guy Kol, Megan Strait, Chloe Fan, Catherine Grevet, and Sarah Elfenbein. 2010. G-nome surfer: a tabletop interface for collaborative exploration of genomic data. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1427–1436.
[54]
Orit Shaer, Ali Mazalek, Brygg Ullmer, and Miriam Konkel. 2013. From big data to insights: opportunities and challenges for TEI in genomics. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction. 109–116.
[55]
Ronglai Shen, Sijian Wang, and Qianxing Mo. 2013. Sparse integrative clustering of multiple omics data sets. The annals of applied statistics 7, 1 (2013), 269.
[56]
Donghee Shin. 2021. The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AI. International Journal of Human-Computer Studies 146 (2021), 102551.
[57]
Keng Siau and Weiyu Wang. 2018. Building trust in artificial intelligence, machine learning, and robotics. Cutter business technology journal 31, 2 (2018), 47–53.
[58]
Desney S Tan, Darren Gergle, Peter Scupelli, and Randy Pausch. 2003. With similar visual angles, larger displays improve spatial performance. In Proceedings of the SIGCHI conference on Human factors in computing systems. 217–224.
[59]
Anthony Tang, Melanie Tory, Barry Po, Petra Neumann, and Sheelagh Carpendale. 2006. Collaborative coupling over tabletop displays. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 1181–1190.
[60]
Mathias Uhlen, Per Oksvold, Linn Fagerberg, Emma Lundberg, Kalle Jonasson, Mattias Forsberg, Martin Zwahlen, Caroline Kampf, Kenneth Wester, Sophia Hober, 2010. Towards a knowledge-based human protein atlas. Nature biotechnology 28, 12 (2010), 1248–1250. https://www.proteinatlas.org/
[61]
Stijn Viaene. 2013. Data scientists aren’t domain experts. IT Professional 15, 6 (2013), 12–17.
[62]
John Wenskovitch and Chris North. 2017. Observation-level interaction with clustering and dimension reduction algorithms. In Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics. 1–6.
[63]
Fumeng Yang, Zhuanyi Huang, Jean Scholtz, and Dustin L Arendt. 2020. How do visual explanations foster end users’ appropriate trust in machine learning?. In Proceedings of the 25th international conference on intelligent user interfaces. 189–201.
[64]
Hubert D Zając, Dana Li, Xiang Dai, Jonathan F Carlsen, Finn Kensing, and Tariq O Andersen. 2023. Clinician-facing AI in the Wild: Taking Stock of the Sociotechnical Challenges and Opportunities for HCI. ACM Transactions on Computer-Human Interaction 30, 2 (2023), 1–39.
[65]
Amy X Zhang, Michael Muller, and Dakuo Wang. 2020. How do data science workers collaborate? roles, workflows, and tools. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1 (2020), 1–23.
Index Terms
- Embodied Machine Learning
Recommendations
Integrating machine learning with knowledge acquisition through direct interaction with domain experts
Knowledge elicitation from experts and empirical machine learning are two distinct approaches to knowledge acquisition with differing and mutually complementary capabilities. Learning apprentices have provided environments in which a knowledge engineer ...
Interpersonal Traits, Complementarity, and Trust in Virtual Collaboration
Trust has been a focus of research on virtual collaboration in distributed teams, e-commerce, e-learning, and telemedicine. Central to several models of trust and virtual collaboration is user's disposition to trust. This construct, however, has ...
Fostering trust in virtual project teams: Towards a design framework grounded in a TrustWorthiness ANtecedents (TWAN) schema
Several collaboration problems in virtual project teams that work in knowledge-intensive contexts can be attributed to a hampered process of interpersonal trust formation. Solutions to trust formation problems need to be based on an understanding of how ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
February 2024
1058 pages
ISBN:9798400704024
DOI:10.1145/3623509
- General Chairs:
- Luigina Ciolfi,
- Trevor Hogan,
- Program Chairs:
- Tanja Döring,
- Tom Jenkins,
- Jelle van Dijk,
- Samuel Huron,
- Zhuying Li,
- Publications Chairs:
- David Coyle,
- Beat Signer
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 February 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Data Availability
Funding Sources
- Natural Sciences and Engineering Research Council of Canada (NSERC)
- Ontario Ministry of Research and Innovation (MRI)
- Social Sciences and Humanities Research Council (SSHRC)
- Canada Foundation for Innovation (CFI)
Conference
TEI '24
Sponsor:
TEI '24: Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction
February 11 - 14, 2024
Cork, Ireland
Acceptance Rates
Overall Acceptance Rate 393 of 1,367 submissions, 29%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 174Total Downloads
- Downloads (Last 12 months)174
- Downloads (Last 6 weeks)41
Reflects downloads up to 12 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format