Integrated Calculators: Moving Calculation into the World
Pages 343 - 355
Abstract
Computing devices commonly act as tools, extending our abilities and shaping how we interact with the world. We investigate one such tool, the calculator, which helps with arithmetic, but also commonly offers specialized functions for conversions, formulas, or graphing. Through an analysis of calculator apps and use cases, we describe limitations of current calculators. Crucially, calculator apps remain detached from tasks, motivating us to explore how to more closely integrate calculation with the world through augmented reality (AR). AR calculators can directly use measurements and numbers from the world in calculations as well as display results of calculations in the world. We provide a conceptual account of calculation in AR, as well as video prototypes that concretize the concept across different scenarios. These examples demonstrate how moving tools like the calculator to AR offers tighter task integration and reduces the work required in translating between the world and computational tools.
Supplemental Material
MP4 File
Concept video showing envisioned use cases of an AR calculator
- Download
- 35.00 MB
References
[1]
Shiri Azenkot, Cynthia L. Bennett, and Richard E. Ladner. 2013. DigiTaps: Eyes-Free Number Entry on Touchscreens with Minimal Audio Feedback. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST ’13). Association for Computing Machinery, New York, NY, USA, 85–90. https://doi.org/10.1145/2501988.2502056
[2]
Arpit Bhatia, Henning Pohl, Teresa Hirzle, Hasti Seifi, and Kasper Hornbæk. 2024. Using the Visual Language of Comics to Alter Sensations in Augmented Reality. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3613904.3642351
[3]
Jonas Blattgerste, Benjamin Strenge, Patrick Renner, Thies Pfeiffer, and Kai Essig. 2017. Comparing Conventional and Augmented Reality Instructions for Manual Assembly Tasks. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments (Island of Rhodes, Greece) (PETRA ’17). Association for Computing Machinery, New York, NY, USA, 75–82. https://doi.org/10.1145/3056540.3056547
[4]
Leonardo Bonanni, Chia-Hsun Lee, and Ted Selker. 2005. Attention-Based Design of Augmented Reality Interfaces. In CHI ’05 Extended Abstracts on Human Factors in Computing Systems (Portland, OR, USA) (CHI EA ’05). Association for Computing Machinery, New York, NY, USA, 1228–1231. https://doi.org/10.1145/1056808.1056883
[5]
Emily C. Bouck, Sara Flanagan, Gauri S. Joshi, Waseem Sheikh, and Dave Schleppenbach. 2011. Speaking Math — A Voice Input, Speech Output Calculator for Students with Visual Impairments. Journal of Special Education Technology 26, 4 (2011), 1–14. https://doi.org/10.1177/016264341102600401
[6]
Mehmet Bulut and Rita Borromeo Ferri. 2023. A systematic literature review on augmented reality in mathematics education. European Journal of Science and Mathematics Education 11 (2023), 556–572. Issue 3. https://doi.org/10.30935/scimath/13124
[7]
Oğuz Turan Buruk and Juho Hamari. 2021. Immersive Video Sketching: Low-Fidelity Extended Reality Prototyping for Everyone. In Proceedings of the 24th International Academic Mindtrek Conference (Tampere/Virtual, Finland) (Academic Mindtrek ’21). Association for Computing Machinery, New York, NY, USA, 165–175. https://doi.org/10.1145/3464327.3464330
[8]
Andreas Bye, Erik Hollnagel, and Tor Steinar Brendeford. 1999. Human–machine function allocation: a functional modelling approach. Reliability Engineering & System Safety 64, 2 (1999), 291–300.
[9]
Paul Cairns, Sameera Wali, and Harold Thimbleby. 2004. Evaluating a Novel Calculator Interface. In Proceedings British Computer Society HCI Conference, Vol. 2. Research Press International, 9–12.
[10]
Bindita Chaudhuri, Leah Perlmutter, Justin Petelka, Philip Garrison, James Fogarty, Jacob O. Wobbrock, and Richard E. Ladner. 2019. GestureCalc: An Eyes-Free Calculator for Touch Screens. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, PA, USA) (ASSETS ’19). ACM, New York, NY, USA, 112–123. https://doi.org/10.1145/3308561.3353783
[11]
Pei-yu Chi, Jen-hao Chen, Hao-hua Chu, and Bing-Yu Chen. 2007. Enabling Nutrition-Aware Cooking in a Smart Kitchen. In CHI ’07 Extended Abstracts on Human Factors in Computing Systems (San Jose, CA, USA) (CHI EA ’07). Association for Computing Machinery, New York, NY, USA, 2333–2338. https://doi.org/10.1145/1240866.1241003
[12]
Marta Civil. 2002. Everyday Mathematics, Mathematicians’ Mathematics, and School Mathematics: Can We Bring Them Together?Journal for Research in Mathematics Education. Monograph 11 (2002), 40–62. http://www.jstor.org/stable/749964
[13]
Helen M. Doerr and Roxana Zangor. 2000. Creating Meaning for and with the Graphing Calculator. Educational Studies in Mathematics 41 (2000), 143–163. https://doi.org/10.1023/A:1003905929557
[14]
Paul Dourish. 2001. Seeking a Foundation for Context-Aware Computing. Human–Computer Interaction 16, 2-4 (2001), 229–241. https://doi.org/10.1207/S15327051HCI16234_07
[15]
Caleb Everett. 2017. Numbers and the making of us: counting and the course of human cultures. Harvard University Press.
[16]
Umer Farooq and Jonathan Grudin. 2016. Human-Computer Integration. Interactions 23, 6 (oct 2016), 26–32. https://doi.org/10.1145/3001896
[17]
Evan M. Glazer and John W. McConnell. 2002. Real-Life Math: Everyday Use of Mathematical Concepts. Greenwood Press, Westport, CT, USA.
[18]
Raphael Grasset, Andreas Dunser, and Mark Billinghurst. 2008. The design of a mixed-reality book: Is it still a real book?. In 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, 99–102. https://doi.org/10.1109/ISMAR.2008.4637333
[19]
Kim Halskov and Caroline Lundqvist. 2021. Filtering and Informing the Design Space: Towards Design-Space Thinking. ACM Trans. Comput.-Hum. Interact. 28, 1, Article 8 (jan 2021), 28 pages. https://doi.org/10.1145/3434462
[20]
Kim Halskov and Rune Nielsen. 2006. Virtual Video Prototyping. Human–Computer Interaction 21, 2 (2006), 199–233. https://doi.org/10.1207/s15327051hci2102_2
[21]
Haosheng Huang, Manuela Schmidt, and Georg Gartner. 2012. Spatial Knowledge Acquisition with Mobile Maps, Augmented Reality and Voice in the Context of GPS-based Pedestrian Navigation: Results from a Field Test. Cartography and Geographic Information Science 39, 2 (2012), 107–116. https://doi.org/10.1559/15230406392107
[22]
Edwin L Hutchins, James D Hollan, and Donald A Norman. 1985. Direct manipulation interfaces. Human–computer interaction 1, 4 (1985), 311–338.
[23]
Seokbin Kang, Ekta Shokeen, Virginia L. Byrne, Leyla Norooz, Elizabeth Bonsignore, Caro Williams-Pierce, and Jon E. Froehlich. 2020. ARMath: Augmenting Everyday Life with Math Learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (, Honolulu, HI, USA, ) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3313831.3376252
[24]
K. Kim, M. Billinghurst, G. Bruder, H. B. Duh, and G. F. Welch. 2018. Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008–2017). IEEE Transactions on Visualization and Computer Graphics 24, 11 (2018), 2947–2962.
[25]
Samuel Kinsley. 2010. Representing ‘Things to Come’: Feeling the Visions of Future Technologies. Environment and Planning A: Economy and Space 42, 11 (2010), 2771–2790. https://doi.org/10.1068/a42371
[26]
David Kirsh. 1995. Complementary Strategies: Why We Use Our Hands When We Think. In Proceedings of the 17th Annual Conference of the Cognitive Science Society. Lawrence Erlbaum Associates, Mahwah, New Jersey, USA, 212–217.
[27]
Jean Lave. 1988. Cognition in practice: Mind, mathematics and culture in everyday life. Cambridge University Press.
[28]
Stephen Law, Brooks Paige, and Chris Russell. 2019. Take a Look Around: Using Street View and Satellite Images to Estimate House Prices. ACM Trans. Intell. Syst. Technol. 10, 5, Article 54 (Sept. 2019), 19 pages. https://doi.org/10.1145/3342240
[29]
Germán Leiva and Michel Beaudouin-Lafon. 2018. Montage: A Video Prototyping System to Reduce Re-Shooting and Increase Re-Usability. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST ’18). Association for Computing Machinery, New York, NY, USA, 675–682. https://doi.org/10.1145/3242587.3242613
[30]
Germán Leiva, Cuong Nguyen, Rubaiat Habib Kazi, and Paul Asente. 2020. Pronto: Rapid Augmented Reality Video Prototyping Using Sketches and Enaction. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376160
[31]
Feiyu Lu and Doug A. Bowman. 2021. Evaluating the Potential of Glanceable AR Interfaces for Authentic Everyday Uses. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, 768–777. https://doi.org/10.1109/VR50410.2021.00104
[32]
Wendy E. Mackay, Anne-Laure Fayard, Laurent Frobert, and Lionel Médini. 1998. Reinventing the Familiar: Exploring an Augmented Reality Design Space for Air Traffic Control. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Los Angeles, California, USA) (CHI ’98). ACM Press/Addison-Wesley Publishing Co., USA, 558–565. https://doi.org/10.1145/274644.274719
[33]
Joanna O. Masingila. 1994. Mathematics Practice in Carpet Laying. Anthropology & Education Quarterly 25, 4 (1994), 430–462. https://doi.org/10.1525/aeq.1994.25.4.04x0531k
[34]
Joanna McGrenere, Ronald M. Baecker, and Kellogg S. Booth. 2002. An Evaluation of a Multiple Interface Design Solution for Bloated Software. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Minneapolis, Minnesota, USA) (CHI ’02). Association for Computing Machinery, New York, NY, USA, 164–170. https://doi.org/10.1145/503376.503406
[35]
Stephen J. Micklo. 1999. Estimation - It’s More than a Guess. Childhood Education 75, 3 (1999), 142–145. https://doi.org/10.1080/00094056.1999.10522001
[36]
Eric Milou. 1999. The Graphing Calculator: A Survey of Classroom Usage. School Science and Mathematics 99, 3 (1999), 133–140. https://doi.org/10.1111/j.1949-8594.1999.tb17461.x
[37]
Peter Mohr, Bernhard Kerbl, Michael Donoser, Dieter Schmalstieg, and Denis Kalkofen. 2015. Retargeting Technical Documentation to Augmented Reality. Association for Computing Machinery, New York, NY, USA, 3337–3346. https://doi.org/10.1145/2702123.2702490
[38]
Franziska Mueller, Florian Bernard, Oleksandr Sotnychenko, Dushyant Mehta, Srinath Sridhar, Dan Casas, and Christian Theobalt. 2018. GANerated Hands for Real-Time 3D Hand Tracking From Monocular RGB. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 49–59. https://doi.org/10.1109/CVPR.2018.00013
[39]
Michael Nebeling and Katy Madier. 2019. 360proto: Making Interactive Virtual Reality & Augmented Reality Prototypes from Paper. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300826
[40]
Zhenxing Niu, Mo Zhou, Le Wang, Xinbo Gao, and Gang Hua. 2016. Ordinal Regression With Multiple Output CNN for Age Estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 4920–4928. https://doi.org/10.1109/CVPR.2016.532
[41]
Donald A Norman. 1988. The Psychology of Everyday Things. Basic books, New York, NY, USA.
[42]
Donald A Norman. 1998. The invisible computer: why good products can fail, the personal computer is so complex, and information appliances are the solution. MIT press, Cambridge, MA, USA.
[43]
Midas Nouwens and Clemens Nylandsted Klokmose. 2018. The Application and Its Consequences for Non-Standard Knowledge Work. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173973
[44]
Terezinha Nunes, Analucia Dias Schliemann, and David William Carraher. 1993. Street mathematics and school mathematics. Cambridge University Press.
[45]
Jef Raskin. 2000. The humane interface: new directions for designing interactive systems. Addison-Wesley Professional.
[46]
Troels A. Rasmussen and Timothy Merritt. 2018. ProjecTables: Augmented CNC tools for sustainable creative practices. International Journal of Architectural Computing 16, 3 (2018), 227–242. https://doi.org/10.1177/1478077118792356
[47]
Umair Rehman and Shi Cao. 2017. Augmented-Reality-Based Indoor Navigation: A Comparative Analysis of Handheld Devices Versus Google Glass. IEEE Transactions on Human-Machine Systems 47, 1 (2017), 140–151. https://doi.org/10.1109/THMS.2016.2620106
[48]
Geoffrey B. Saxe. 1988. Candy Selling and Math Learning. Educational Researcher 17, 6 (1988), 14–21. https://doi.org/10.3102/0013189X017006014
[49]
Mike Scaife and Yvonne Rogers. 1996. External cognition: how do graphical representations work?International journal of human-computer studies 45, 2 (1996), 185–213.
[50]
Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative Effectiveness of Augmented Reality in Object Assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI ’03). Association for Computing Machinery, New York, NY, USA, 73–80. https://doi.org/10.1145/642611.642626
[51]
Ryosuke Tanno, Takumi Ege, and Keiji Yanai. 2018. AR DeepCalorieCam V2: Food Calorie Estimation with CNN and AR-based Actual Size Estimation. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (Tokyo, Japan) (VRST ’18). ACM, New York, NY, USA, Article 46, 2 pages. https://doi.org/10.1145/3281505.3281580
[52]
Harold Thimbleby. 2000. Calculators are needlessly bad. International Journal of Human-Computer Studies 52, 6 (2000), 1031–1069. https://doi.org/10.1006/ijhc.1999.0341
[53]
Frederic Valle-Tourangeau, Miroslav Sirota, and Gaëlle Villejoubert. 2013. Reducing the impact of math anxiety on mental arithmetic: The importance of distributed cognition. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 35. 3615–3620.
[54]
Frédéric Vallée-Tourangeau and Gaëlle Villejoubert. 2013. Naturalising Problem Solving. In Cognition Beyond the Brain: Computation, Interactivity and Human Artifice, Stephen J. Cowley and Frédéric Vallée-Tourangeau (Eds.). Springer London, London, 241–253. https://doi.org/10.1007/978-1-4471-5125-8_13
[55]
L. Vertelney. 1989. Using Video to Prototype User Interfaces. SIGCHI Bull. 21, 2 (oct 1989), 57–61. https://doi.org/10.1145/70609.70615
[56]
Tri-Viet Vo, Minh Nguyen, and Huy Le. 2018. Augmented Reality on Mobile Platform: A New Way to Instantly View and Display Foreign Currency Exchange Rate. In 2018 International Conference on Image and Vision Computing New Zealand (IVCNZ). IEEE, 1–5. https://doi.org/10.1109/IVCNZ.2018.8634773
[57]
Christian Weichel, Jason Alexander, Abhijit Karnik, and Hans Gellersen. 2015. SPATA: Spatio-Tangible Tools for Fabrication-Aware Design. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (Stanford, California, USA) (TEI ’15). Association for Computing Machinery, New York, NY, USA, 189–196. https://doi.org/10.1145/2677199.2680576
[58]
Galit P. Wellner. 2015. A Postphenomenological Inquiry of Cell Phones: Genealogies, Meanings, and Becoming. Lexington Books.
[59]
Pierre Wellner. 1991. The DigitalDesk Calculator: Tangible Manipulation on a Desk Top Display. In Proceedings of the 4th Annual ACM Symposium on User Interface Software and Technology (Hilton Head, South Carolina, USA) (UIST ’91). ACM, New York, NY, USA, 27–33. https://doi.org/10.1145/120782.120785
[60]
Richmond Y. Wong and Vera Khovanskaya. 2018. Speculative Design in HCI: From Corporate Imaginations to Critical Orientations. In New Directions in Third Wave Human-Computer Interaction: Volume 2 - Methodologies, Michael Filimowicz and Veronika Tzankova (Eds.). Springer International Publishing, Cham, 175–202. https://doi.org/10.1007/978-3-319-73374-6_10
[61]
Richmond Y. Wong and Deirdre K. Mulligan. 2016. When a Product Is Still Fictional: Anticipating and Speculating Futures through Concept Videos. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS ’16). Association for Computing Machinery, New York, NY, USA, 121–133. https://doi.org/10.1145/2901790.2901801
[62]
J.D. Zamfirescu-Pereira, David Sirkin, David Goedicke, RAY LC, Natalie Friedman, Ilan Mandel, Nikolas Martelaro, and Wendy Ju. 2021. Fake It to Make It: Exploratory Prototyping in HRI. Association for Computing Machinery, New York, NY, USA, 19–28. https://doi.org/10.1145/3434074.3446909
[63]
Stefanie Zollmann, Raphaël Grasset, Tobias Langlotz, Wei Hong Lo, Shohei Mori, and Holger Regenbrecht. 2020. Visualization Techniques in Augmented Reality: A Taxonomy, Methods and Patterns. IEEE Transactions on Visualization and Computer Graphics 27, 9 (2020), 1–20. https://doi.org/10.1109/TVCG.2020.2986247
Index Terms
- Integrated Calculators: Moving Calculation into the World
Recommendations
Investigating the balance between virtuality and reality in mobile mixed reality UI design: user perception of an augmented city
NordiCHI '14: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, FoundationalExamples of mixed reality mobile applications and research combining virtual and real world data in the same view have emerged during recent years. However, currently there is little knowledge of users' perceptions comparing the role of virtual and real ...
Mixed reality in virtual world teleconferencing
VR '10: Proceedings of the 2010 IEEE Virtual Reality ConferenceIn this paper we present a Mixed Reality (MR) teleconferencing application based on Second Life (SL) and the OpenSim virtual world. Augmented Reality (AR) techniques are used for displaying virtual avatars of remote meeting participants in real physical ...
Dual Phone AR: Using a Second Phone as a Controller for Mobile Augmented Reality
UIST '20 Adjunct: Adjunct Proceedings of the 33rd Annual ACM Symposium on User Interface Software and TechnologyMobile Augmented Reality applications have become increasingly popular, however the possible interactions with AR content are largely limited to on-screen gestures and spatial movement. There has been a renewed interest in designing interaction methods ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2024
3616 pages
ISBN:9798400705830
DOI:10.1145/3643834
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution-ShareAlike International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
DIS '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,158 of 4,684 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 216Total Downloads
- Downloads (Last 12 months)216
- Downloads (Last 6 weeks)60
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in