research-article

Open access

The Impact of Expertise in the Loop for Exploring Machine Rationality

Authors:

Andreas Martin ButzAuthors Info & Claims

IUI '23: Proceedings of the 28th International Conference on Intelligent User Interfaces

Pages 307 - 321

https://doi.org/10.1145/3581641.3584040

Published: 27 March 2023 Publication History

All formats PDF

Abstract

Human-in-the-loop optimization utilizes human expertise to guide machine optimizers iteratively and search for an optimal solution in a solution space. While prior empirical studies mainly investigated novices, we analyzed the impact of the levels of expertise on the outcome quality and corresponding subjective satisfaction. We conducted a study (N=60) in text, photo, and 3D mesh optimization contexts. We found that novices can achieve an expert level of quality performance, but participants with higher expertise led to more optimization iteration with more explicit preference while keeping satisfaction low. In contrast, novices were more easily satisfied and terminated faster. Therefore, we identified that experts seek more diverse outcomes while the machine reaches optimal results, and the observed behavior can be used as a performance indicator for human-in-the-loop system designers to improve underlying models. We inform future research to be cautious about the impact of user expertise when designing human-in-the-loop systems.

References

[1]

Paul Anand. 1987. Are the preference axioms really rational?Theory and Decision 23, 2 (Sep 1987), 189–214. https://doi.org/10.1007/BF00126305

[2]

Jukka Arvo, Antti Euranto, Lauri Jarvenpaa, Teijo Lehtonen, and Timo Knuutila. 2015. 3D Mesh Simplification – A survey of algorithms and CAD model simplification tests. University of Turku, Turku, Finland. http://urn.fi/URN:ISBN:978-951-29-6202-0

[3]

Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy. 2020. BOTORCH: A Framework for Efficient Monte-Carlo Bayesian Optimization. In Proceedings of the 34th International Conference on Neural Information Processing Systems(Vancouver, BC, Canada) (NIPS’20). Curran Associates Inc., Red Hook, NY, USA, Article 1807, 15 pages.

[4]

Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S. Weld, Walter S. Lasecki, and Eric Horvitz. 2019. Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff. Proceedings of the AAAI Conference on Artificial Intelligence 33, 0101 (Jul 2019), 2429–2437. https://doi.org/10.1609/aaai.v33i01.33012429

Digital Library

[5]

H. G. Barrow, J. M. Tenenbaum, R. C. Bolles, and H. C. Wolf. 1977. Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching. https://apps.dtic.mil/sti/citations/ADA458355

[6]

Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2014. Fitting Linear Mixed-Effects Models using lme4. https://doi.org/10.48550/ARXIV.1406.5823

[7]

Alessio Benavoli, Dario Azzimonti, and Dario Piga. 2021. Choice functions based multi-objective Bayesian optimisation. https://doi.org/10.48550/arXiv.2110.08217

[8]

Mario Botsch, Leif Kobbelt, Mark Pauly, Pierre Alliez, and Bruno Lévy. 2010. Polygon mesh processing. CRC press, USA.

[9]

Lyle Bourne, James Kole, and Alice Healy. 2014. Expertise: defined, described, explained. Frontiers in Psychology 5 (2014), 186. https://doi.org/10.3389/fpsyg.2014.00186

[10]

Eduard Brandstätter, Gerd Gigerenzer, and Ralph Hertwig. 2006. The Priority Heuristic: Making Choices Without Trade-Offs. Psychological review 113, 2 (Apr 2006), 409–432. https://doi.org/10.1037/0033-295X.113.2.409

[11]

Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian Interactive Optimization Approach to Procedural Animation Design. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation(Madrid, Spain) (SCA ’10). Eurographics Association, Goslar, DEU, 103–112. https://dl.acm.org/doi/10.5555/1921427.1921443

Digital Library

[12]

Eric Brochu, Nando de Freitas, and Abhijeet Ghosh. 2007. Active preference learning with discrete choice data. In Proceedings of the 20th International Conference on Neural Information Processing Systems(NIPS’07). Curran Associates Inc., Red Hook, NY, USA, 409–416. https://dl.acm.org/doi/abs/10.5555/2981562.2981614

Digital Library

[13]

Liwei Chan, Yi-Chi Liao, George B. Mo, John J. Dudley, Chun-Lien Cheng, Per Ola Kristensson, and Antti Oulasvirta. 2022. Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3491102.3501850

Digital Library

[14]

Donald L. Cohn. 2013. Probability. Springer, New York, NY. 307–371 pages. https://doi.org/10.1007/978-1-4614-6956-8_10

[15]

Harry Collins. 2013. Three dimensions of expertise. Phenomenology and the Cognitive Sciences 12, 2 (June 2013), 253–273. https://doi.org/10.1007/s11097-011-9203-5

[16]

Massimiliano Corsini, Mohamed-Chaker Larabi, Guillaume Lavoué, Oldřich Petřík, Libor Váša, and Kai Wang. 2013. Perceptual Metrics for Static and Dynamic Triangle Meshes. Computer Graphics Forum 32, 1 (2013), 101–125. https://doi.org/10.1111/cgf.12001

[17]

Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. 2021. Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.). Vol. 34. Curran Associates, Inc., Red Hook, NY, USA, 2187–2200. https://proceedings.neurips.cc/paper/2021/file/11704817e347269b7254e744b5e22dac-Paper.pdf

[18]

Berkeley J. Dietvorst, Joseph P. Simmons, and Cade Massey. 2015. Algorithm aversion: people erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology. General 144, 1 (Feb 2015), 114–126. https://doi.org/10.1037/xge0000033

[19]

Upol Ehsan, Q. Vera Liao, Michael Muller, Mark O. Riedl, and Justin D. Weisz. 2021. Expanding Explainability: Towards Social Transparency in AI systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI’21). Association for Computing Machinery, New York, NY, USA, 1–19. https://doi.org/10.1145/3411764.3445188

Digital Library

[20]

Roger Ferrod, Federica Cena, Luigi Di Caro, Dario Mana, and Rossana Grazia Simeoni. 2021. Identifying users’ domain expertise from dialogues. In Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization(UMAP ’21). Association for Computing Machinery, New York, NY, USA, 29–34. https://doi.org/10.1145/3450614.3461683

Digital Library

[21]

Michael Garland and Paul S. Heckbert. 1998. Simplifying surfaces with color and texture using quadric error metrics. In Proceedings of the conference on Visualization ’98(VIS ’98). IEEE Computer Society Press, Washington, DC, USA, 263–269. https://doi.org/10.5555/288216.288280

Digital Library

[22]

S.K. Garrett, B.S. Caldwell, E.C. Harris, and M.C. Gonzalez. 2009. Six dimensions of expertise: a more comprehensive definition of cognitive expertise for team coordination. Theoretical Issues in Ergonomics Science 10, 2 (March 2009), 93–105. https://doi.org/10.1080/14639220802059190

[23]

Gerd Gigerenzer and Henry Brighton. 2009. Homo heuristicus: why biased minds make better inferences. Topics in Cognitive Science 1, 1 (Jan 2009), 107–143. https://doi.org/10.1111/j.1756-8765.2008.01006.x

[24]

Javier González, Zhenwen Dai, Andreas Damianou, and Neil D. Lawrence. 2017. Preferential Bayesian Optimization. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 (Sydney, NSW, Australia) (ICML’17). JMLR.org, Australia, 1282–1291. https://doi.org/10.5555/3305381.3305514

Digital Library

[25]

William M. Grove and Paul E. Meehl. 1996. Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. Psychology, Public Policy, and Law 2, 2 (1996), 293–323. https://doi.org/10.1037/1076-8971.2.2.293

[26]

Till Grüne. 2004. The Problems of Testing Preference Axioms with Revealed Preference Theory. Analyse & Kritik 26, 2 (Nov 2004), 382–397. https://doi.org/10.1515/auk-2004-0204

[27]

Daniel M. Hausman. 2011. Preference, Value, Choice, and Welfare. Cambridge University Press, New York, USA.

[28]

Daniel Holliday, Stephanie Wilson, and Simone Stumpf. 2016. User Trust in Intelligent Systems: A Journey Over Time. In Proceedings of the 21st International Conference on Intelligent User Interfaces (Sonoma, California, USA) (IUI ’16). Association for Computing Machinery, New York, NY, USA, 164–168. https://doi.org/10.1145/2856767.2856811

Digital Library

[29]

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The Curious Case of Neural Text Degeneration. https://doi.org/10.48550/ARXIV.1904.09751

[30]

Sheena S Iyengar, Rachael E Wells, and Barry Schwartz. 2006. Doing better but feeling worse: Looking for the “best” job undermines satisfaction. Psychological Science 17, 2 (2006), 143–150. https://doi.org/10.1111/j.1467-9280.2006.01677.x

[31]

Wenzel Jakob, Marco Tarini, Daniele Panozzo, and Olga Sorkine-Hornung. 2015. Instant Field-Aligned Meshes. ACM Transactions on Graphics 34, 6 (Oct. 2015), 189:1–189:15. https://doi.org/10.1145/2816795.2818078

Digital Library

[32]

Dietmar Jannach, Markus Zanker, Alexander Felfernig, and Gerhard Friedrich. 2010. Recommender Systems: An Introduction. Cambridge University Press, New York, USA.

[33]

Zhiyuan Jerry Lin, Raul Astudillo, Peter Frazier, and Eytan Bakshy. 2022. Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics(Proceedings of Machine Learning Research, Vol. 151), Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera (Eds.). PMLR, Virtual, 4235–4258. https://proceedings.mlr.press/v151/jerry-lin22a.html

[34]

Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein. 2021. Noise. HarperCollins UK, UK.

[35]

René F. Kizilcec. 2016. How Much Information? Effects of Transparency on Trust in an Algorithmic Interface. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI ’16). Association for Computing Machinery, New York, NY, USA, 2390–2395. https://doi.org/10.1145/2858036.2858402

Digital Library

[36]

Patrick M. Knupp. 2000. Achieving finite element mesh quality via optimization of the Jacobian matrix norm and associated quantities. Part I—a framework for surface mesh optimization. Internat. J. Numer. Methods Engrg. 48, 3 (2000), 401–420. https://doi.org/10.1002/(SICI)1097-0207(20000530)48:3<401::AID-NME880>3.0.CO;2-D

[37]

Jari Korhonen and Junyong You. 2012. Peak signal-to-noise ratio revisited: Is simple beautiful?. In 2012 Fourth International Workshop on Quality of Multimedia Experience. IEEE, Melbourne, VIC, Australia, 37–38. https://doi.org/10.1109/QoMEX.2012.6263880

[38]

Ben Kotzee and Jp Smit. 2018. Two Social Dimensions of Expertise. In Education and Expertise. John Wiley & Sons, Ltd, New Jersey, USA, 99–116. https://doi.org/10.1002/9781119527268.ch5

[39]

Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2016. SelPh: Progressive Learning and Support of Manual Photo Color Enhancement. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 2520–2532. https://doi.org/10.1145/2858036.2858111

Digital Library

[40]

Yuki Koyama, Issei Sato, and Masataka Goto. 2020. Sequential gallery for interactive visual design optimization. ACM Transactions on Graphics 39, 4 (July 2020), 88:88:1–88:88:12. https://doi.org/10.1145/3386569.3392444

Digital Library

[41]

Yuki Koyama, Issei Sato, Daisuke Sakamoto, and Takeo Igarashi. 2017. Sequential Line Search for Efficient Visual Design Optimization by Crowds. ACM Trans. Graph. 36, 4, Article 48 (jul 2017), 11 pages. https://doi.org/10.1145/3072959.3073598

Digital Library

[42]

Vijay Krishna and John Morgan. 2001. A Model of Expertise. The Quarterly Journal of Economics 116, 2 (May 2001), 747–775. https://doi.org/10.1162/00335530151144159

[43]

Alexandra Kuznetsova, Per B. Brockhoff, and Rune H. B. Christensen. 2017. lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software 82, 13 (2017), 1–26. https://doi.org/10.18637/jss.v082.i13

[44]

Michael D. Lee, Mark Steyvers, Mindy de Young, and Brent Miller. 2012. Inferring Expertise in Knowledge and Prediction Ranking Tasks. Topics in Cognitive Science 4, 1 (2012), 151–163. https://doi.org/10.1111/j.1756-8765.2011.01175.x

[45]

Thibault Lescoat, Hsueh-Ti Derek Liu, Jean-Marc Thiery, Alec Jacobson, Tamy Boubekeur, and Maks Ovsjanikov. 2020. Spectral Mesh Simplification. Computer Graphics Forum 39, 2 (2020), 315–324. https://doi.org/10.1111/cgf.13932

[46]

Benjamin Letham, Brian Karrer, Guilherme Ottoni, and Eytan Bakshy. 2019. Constrained Bayesian Optimization with Noisy Experiments. Bayesian Analysis 14, 2 (Jun 2019), 495–519. https://doi.org/10.1214/18-BA1110

[47]

Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74–81. https://aclanthology.org/W04-1013

[48]

J. Marks, B. Andalman, P. A. Beardsley, W. Freeman, S. Gibson, J. Hodgins, T. Kang, B. Mirtich, H. Pfister, W. Ruml, K. Ryall, J. Seims, and S. Shieber. 1997. Design Galleries: A General Approach to Setting Parameters for Computer Graphics and Animation. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques(SIGGRAPH ’97). ACM Press/Addison-Wesley Publishing Co., USA, 389–400. https://doi.org/10.1145/258734.258887

Digital Library

[49]

Petrus Mikkola, Milica Todorović, Jari Järvi, Patrick Rinke, and Samuel Kaski. 2020. Projective Preferential Bayesian Optimization. In International Conference on Machine Learning. PMLR, MLResearchPress, Online, 6884–6892. https://doi.org/10.5555/3524938.3525577

Digital Library

[50]

Arthur I. Miller. 2019. The Artist in the Machine: The World of AI-Powered Creativity. MIT Press, Cambridge, Massachusetts, USA.

[51]

Robert M. Monarch. 2021. Human-in-the-Loop Machine Learning: Active Learning and Annotation for Human-centered AI. Simon and Schuster, USA.

[52]

Jeroen Ooge and Katrien Verbert. 2021. Trust in Prediction Models: a Mixed-Methods Pilot Study on the Impact of Domain Expertise. https://doi.org/10.48550/arXiv.2109.08183 arXiv:arXiv:2109.08183

[53]

Changkun Ou, Daniel Buschek, Sven Mayer, and Andreas Butz. 2022. The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures. In Proceedings of Mensch Und Computer 2022(MuC ’22). Association for Computing Machinery, Darmstadt, Germany, 1–11. https://doi.org/10.1145/3543758.3543761

Digital Library

[54]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 311–318. https://doi.org/10.3115/1073083.1073135

Digital Library

[55]

Vilfredo Pareto. 1912. Manuel d’économie politique. Bull. Amer. Math. Soc 18, 462-474 (1912), 3.

[56]

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. In Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32 (Beijing, China) (ICML’14). JMLR.org, Beijing, China, II–1278–II–1286. https://proceedings.mlr.press/v32/rezende14.html

Digital Library

[57]

Robert P. Rooderkerk, Harald J. Van Heerde, and Tammo H.A. Bijmolt. 2011. Incorporating Context Effects into a Choice Model. Journal of Marketing Research 48, 4 (Aug 2011), 767–780. https://doi.org/10.1509/jmkr.48.4.767

[58]

David Schmidtz. 2004. Satisficing as a Humanly Rational Strategy. In Satisficing and Maximizing: Moral Theorists on Practical Reason, Michael Byron (Ed.). Cambridge University Press, New York, USA, 30–59. https://doi.org/10.1017/CBO9780511617058.003

[59]

Adriana Schulz, Harrison Wang, Eitan Grinspun, Justin Solomon, and Wojciech Matusik. 2018. Interactive exploration of design trade-offs. ACM Transactions on Graphics 37, 4 (Jul 2018), 131:1–131:14. https://doi.org/10.1145/3197517.3201385

Digital Library

[60]

Barry Schwartz, Andrew Ward, John Monterosso, Sonja Lyubomirsky, Katherine White, and Darrin R. Lehman. 2002. Maximizing versus satisficing: Happiness is a matter of choice.Journal of Personality and Social Psychology 83, 5 (Nov 2002), 1178–1197. https://doi.org/10.1037/0022-3514.83.5.1178

[61]

Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas. 2016. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proc. IEEE 104, 1 (Jan. 2016), 148–175. https://doi.org/10.1109/JPROC.2015.2494218

[62]

Eero Siivola, Akash Kumar Dhaka, Michael Riis Andersen, Javier González, Pablo García Moreno, and Aki Vehtari. 2021. Preferential Batch Bayesian Optimization. In 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP). IEEE, Gold Coast, Australia, 1–6. https://doi.org/10.1109/MLSP52302.2021.9596494

[63]

Herbert A. Simon. 1955. A Behavioral Model of Rational Choice. The Quarterly Journal of Economics 69, 1 (Feb. 1955), 99–118. https://doi.org/10.2307/1884852

[64]

Edwin Simpson, Yang Gao, and Iryna Gurevych. 2020. Interactive Text Ranking with Bayesian Optimization: A Case Study on Community QA and Summarization. Transactions of the Association for Computational Linguistics 8 (12 2020), 759–775. https://doi.org/10.1162/tacl_a_00344

[65]

Louis L. Thurstone. 1927. A law of comparative judgment. Psychological Review 34, 4 (1927), 273–286. https://doi.org/10.1037/h0070288

[66]

Jeffrey W. Treem and Paul M. Leonardi. 2016. Expertise, Communication, and Organizing. Oxford University Press, New York, NY, USA.

[67]

Amos Tversky and Daniel Kahneman. 1974. Judgment under Uncertainty: Heuristics and Biases. Science 185, 4157 (Sept. 1974), 1124–1131. https://doi.org/10.1126/science.185.4157.1124

[68]

Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (Apr 2004), 600–612. https://doi.org/10.1109/TIP.2003.819861

Digital Library

[69]

James T. Wilson, Riccardo Moriconi, Frank Hutter, and Marc Peter Deisenroth. 2017. The reparameterization trick for acquisition functions. https://doi.org/10.48550/ARXIV.1712.00424

[70]

Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 143–146. https://doi.org/10.1145/1978942.1978963

Digital Library

[71]

Yonghao Yue, Yuki Koyama, Issei Sato, and Takeo Igarashi. 2021. User interfaces for high-dimensional design problems: from theories to implementations. In ACM SIGGRAPH 2021 Courses(SIGGRAPH ’21). Association for Computing Machinery, New York, NY, USA, 1–34. https://doi.org/10.1145/3450508.3464551

Digital Library

[72]

Yijun Zhou, Yuki Koyama, Masataka Goto, and Takeo Igarashi. 2021. Interactive Exploration-Exploitation Balancing for Generative Melody Composition. In 26th International Conference on Intelligent User Interfaces(IUI ’21). Association for Computing Machinery, New York, NY, USA, 43–47. https://doi.org/10.1145/3397481.3450663

Digital Library

Cited By

Chiossi FOu CMayer S(2023)Exploring Physiological Correlates of Visual Complexity Adaptation: Insights from EDA, ECG, and EEG Data for Adaptation Evaluation in VR Adaptive SystemsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585624(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585624

Index Terms

The Impact of Expertise in the Loop for Exploring Machine Rationality
1. Computing methodologies
  1. Machine learning
    1. Learning settings
      1. Active learning settings
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction paradigms

Recommendations

The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures
MuC '22: Proceedings of Mensch und Computer 2022

Interactive AI systems increasingly employ a human-in-the-loop strategy. This creates new challenges for the HCI community when designing such systems. We reveal and investigate some of these challenges in a case study with an industry partner, and ...
Expertise transfer and complex problems
Special issue: 1969-1999, the 30th anniversary

Acquiring knowledge from a human expert is a major problem when building a knowledge-based system. Aquinas, an expanded version of the Expertise Transfer System (ETS), is a knowledge-acquisition workbench that combines ideas from psychology and ...
Extending domain expertise through student modeling

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '23: Proceedings of the 28th International Conference on Intelligent User Interfaces

March 2023

972 pages

ISBN:9798400701061

DOI:10.1145/3581641

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 March 2023

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IUI '23

Sponsor:

IUI '23: 28th International Conference on Intelligent User Interfaces

March 27 - 31, 2023

NSW, Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
335
Total Downloads

Downloads (Last 12 months)224
Downloads (Last 6 weeks)16

Reflects downloads up to 14 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chiossi FOu CMayer S(2023)Exploring Physiological Correlates of Visual Complexity Adaptation: Insights from EDA, ECG, and EEG Data for Adaptation Evaluation in VR Adaptive SystemsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585624(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585624

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents