research-article

Trade-offs for Substituting a Human with an Agent in a Pair Programming Context: The Good, the Bad, and the Ugly

Authors:

Sandeep Kaur Kuttal,

Peter RobeAuthors Info & Claims

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Article No.: 243, Pages 1 - 20

https://doi.org/10.1145/3411764.3445659

Published: 07 May 2021 Publication History

Abstract

Pair programming has a documented history of benefits, such as increased code quality, productivity, self-efficacy, knowledge transfer, and reduced gender gap. Research uncovered problems with pair programming related to scheduling, collocating, role imbalance, and power dynamics. We investigated the trade-offs of substituting a human with an agent to simultaneously provide benefits and alleviate obstacles in pair programming. We conducted gender-balanced studies with human-human pairs in a remote lab with 18 programmers and Wizard-of-Oz studies with 14 programmers, then analyzed results quantitatively and qualitatively. Our comparative analysis of the two studies showed no significant differences in productivity, code quality, and self-efficacy. Further, agents facilitated knowledge transfer; however, unlike humans, agents were unable to provide logical explanations or discussions. Human partners trusted and showed humility towards agents. Our results demonstrate that agents can act as effective pair programming partners and open the way towards new research on conversational agents for programming.

Supplementary Material

MP4 File (3411764.3445659_videopreview.mp4)

Preview video

Download
10.78 MB

References

[1]

Piotr D. Adamczyk and Brian P. Bailey. 2004. If Not Now, When? The Effects of Interruption at Different Moments within Task Execution. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI ’04). Association for Computing Machinery, New York, NY, USA, 271–278. https://doi.org/10.1145/985692.985727

Digital Library

[2]

Maryam Alavi. 1994. Computer-Mediated Collaborative Learning: An Empirical Evaluation. MIS Quarterly 18, 2 (1994), 159–174. http://www.jstor.org/stable/249763

Digital Library

[3]

S. Ali, L. C. Briand, H. Hemmati, and R. K. Panesar-Walawege. 2010. A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation. IEEE Transactions on Software Engineering 36, 6 (2010), 742–762.

Digital Library

[4]

Ali Alkhatlan and Jugal Kalita. 2018. Intelligent Tutoring Systems: A Comprehensive Historical Survey with Recent Developments. arxiv:1812.09628 [cs.HC]

[5]

Teresa M. Amabile and Michael G. Pratt. 2016. The dynamic componential model of creativity and innovation in organizations: Making progress, making meaning. Research in Organizational Behavior 36 (2016), 157 – 183. https://doi.org/10.1016/j.riob.2016.10.001

[6]

Amazon. 2020. Virtual Assistant Amazon Alexa. https://developer.amazon.com/en-US/alexa

[7]

Apple. 2020. Virtual Assistant Apple Siri. https://www.apple.com/siri/

[8]

E. Arisholm, H. Gallis, T. Dybå, and D. I. K. Sjoberg. 2007. Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise. IEEE Transactions on Software Engineering 33, 2 (2007), 65–86.

Digital Library

[9]

Michael Armstrong. 2012. Armstrong’s handbook of reward management practice: Improving performance through reward (12 ed.). Kogan Page Publishers, London.

[10]

Zahra Ashktorab, Mohit Jain, Q. Vera Liao, and Justin D. Weisz. 2019. Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, Article 254, 12 pages. https://doi.org/10.1145/3290605.3300484

Digital Library

[11]

Prashant Baheti, Edward F. Gehringer, and P. David Stotts. 2002. Exploring the Efficacy of Distributed Pair Programming. In Proceedings of the Second XP Universe and First Agile Universe Conference on Extreme Programming and Agile Methods - XP/Agile Universe 2002. Springer-Verlag, Berlin, Heidelberg, 208–220.

[12]

Ryan S.J.d. Baker. 2007. Modeling and Understanding Students’ off-Task Behavior in Intelligent Tutoring Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 1059–1068. https://doi.org/10.1145/1240624.1240785

Digital Library

[13]

A. Bandura. 1986. Social Foundations of Thought and Action: A Social Cognitive Theory. Prentice-Hall, Michigan. https://books.google.com/books?id=HJhqAAAAMAAJ

[14]

Alejandro Barredo Arrieta, Natalia DÃaz-RodrÃguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58(2020), 82 – 115. http://www.sciencedirect.com/science/article/pii/S1566253519308103

Digital Library

[15]

A. Belshee. 2005. Promiscuous pairing and beginner’s mind: embrace inexperience [agile programming]. In Agile Development Conference (ADC’05). Agile Development Conference, Denver, Colorado, 125–131. https://doi.org/10.1109/ADC.2005.37

Digital Library

[16]

Jeannette Bennett, Kendra Cooper, and Lirong Dai. 2010. Aspect-oriented model-driven skeleton code generation: A graph-based transformation approach. Science of Computer Programming 75, 8 (2010), 689–725.

Digital Library

[17]

Timothy Bickmore and Justine Cassell. 2001. Relational Agents: A Model and Implementation of Building User Trust. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Seattle, Washington, USA) (CHI ’01). ACM, New York, NY, USA, 396–403. https://doi.org/10.1145/365024.365304

Digital Library

[18]

Jay Bradley, David Benyon, Oli Mival, and Nick Webb. 2010. Wizard of Oz Experiments and Companion Dialogues. In Proceedings of the 24th BCS Interaction Specialist Group Conference (Dundee, United Kingdom) (BCS ’10). BCS Learning & Development Ltd., Swindon, GBR, 117–123.

[19]

Sheryl Brahnam and Antonella De Angeli. 2012. Gender affordances of conversational agents. Interacting with Computers 24, 3 (04 2012), 139–153. https://doi.org/10.1016/j.intcom.2012.05.001

Digital Library

[20]

Virginia Braun and Victoria Clarke. 2006. Using Thematic Analysis in Psychology. Qualitative research in psychology 3 (01 2006), 77–101. https://doi.org/10.1191/1478088706qp063oa

[21]

Tim Brown. 2009. Change by Design: How Design Thinking Transforms Organizations and Inspires Innovation. HarperBusiness, New York.

[22]

Margaret Burnett, Anicia Peters, Charles Hill, and Noha Elarief. 2016. Finding Gender-Inclusiveness Software Issues with GenderMag: A Field Investigation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, San Jose, 2586–2598.

Digital Library

[23]

Margaret Burnett, Simone Stumpf, Jamie Macbeth, Stephann Makri, Laura Beckwith, Irwin Kwan, Anicia Peters, and William Jernigan. 2016. GenderMag: A Method for Evaluating Software’s Gender Inclusiveness. Interacting with Computers forthcoming (01 2016). https://doi.org/10.1093/iwc/iwv046

[24]

Margaret M. Burnett. 2020. GenderMag. http://gendermag.org/

[25]

Ramón Burri. 2018. Improving user trust towards conversational chatbot interfaces with voice output. Master’s thesis. KTH, School of Electrical Engineering and Computer Science (EECS).

[26]

Judith Butler. 1999. Revisiting Bodies and Pleasures. Theory, Culture & Society 16, 2 (1999), 11–20. https://doi.org/10.1177/02632769922050520

[27]

Lan Cao, Kannan Mohan, Peng Xu, and Balasubramaniam Ramesh. 2004. How Extreme Does Extreme Programming Have to Be? Adapting XP Practices to Large-Scale Projects. In Proceedings of the Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS’04) - Track 3 - Volume 3(HICSS ’04). IEEE Computer Society, USA, 30083.3.

[28]

Robert Cartwright, Eric Allen, and Charles Reis. 2002. Production Programming in the Classroom. ACM SIGCSE Bulletin 35 (11 2002). https://doi.org/10.1145/611892.611940

Digital Library

[29]

Mehmet Celepkolu and Kristy Elizabeth Boyer. 2018. The Importance of Producing Shared Code Through Pair Programming. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (Baltimore, Maryland, USA) (SIGCSE ’18). Association for Computing Machinery, New York, NY, USA, 765–770. https://doi.org/10.1145/3159450.3159506

Digital Library

[30]

Mehmet Celepkolu and Kristy Elizabeth Boyer. 2018. Thematic Analysis of Students’ Reflections on Pair Programming in CS1. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (Baltimore, Maryland, USA) (SIGCSE ’18). Association for Computing Machinery, New York, NY, USA, 771–776. https://doi.org/10.1145/3159450.3159516

Digital Library

[31]

Christopher P Cerasoli, Jessica M Nicklin, and Michael T Ford. 2014. Intrinsic motivation and extrinsic incentives jointly predict performance: A 40-year meta-analysis.Psychological bulletin 140, 4 (2014), 980.

[32]

Ruth W Chabay and Jill H Larkin. 2020. Computer assisted instruction and intelligent tutoring systems: Shared goals and complementary approaches. Routledge, Abingdon, United Kingdom.

[33]

Gary Charness and Uri Gneezy. 2012. Strong Evidence for Gender Differences in Risk Taking. Journal of Economic Behavior & Organization 83, 1(2012), 50–58.

[34]

J. Y. C. Chen and M. J. Barnes. 2014. Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues. IEEE Transactions on Human-Machine Systems 44, 1 (2014), 13–29.

[35]

K. S. Choi. 2013. Evaluating Gender Significance within a Pair Programming Context. In 2013 46th Hawaii International Conference on System Sciences. IEEE, Hawaii, 4817–4825. https://doi.org/10.1109/HICSS.2013.209

Digital Library

[36]

Jan Chong and Tom Hurlbutt. 2007. The Social Dynamics of Pair Programming. In Proceedings of the 29th International Conference on Software Engineering(ICSE ’07). IEEE Computer Society, USA, 354–363. https://doi.org/10.1109/ICSE.2007.87

Digital Library

[37]

Alistair Cockburn and Laurie Williams. 2001. The Costs and Benefits of Pair Programming. Addison-Wesley Longman Publishing Co., Inc., USA, 223–243.

Digital Library

[38]

Deborah R. Compeau and Christopher A. Higgins. 1995. Computer Self-Efficacy: Development of a Measure and Initial Test. MIS Q. 19, 2 (June 1995), 189–211. https://doi.org/10.2307/249688

Digital Library

[39]

Pedro Costa and Luísa Ribas. 2019. AI becomes her: Discussing gender and artificial intelligence.Technoetic Arts: A Journal of Speculative Research 17, 1/2(2019), 171 – 193.

[40]

Sally Jo Cunningham, Annika Hinze, and David M. Nichols. 2016. Supporting Gender-Neutral Digital Library Creation: A Case Study Using the GenderMag Toolkit. In Digital Libraries: Knowledge, Information, and Data in an Open Access Society, Atsuyuki Morishima, Andreas Rauber, and Chern Li Liew (Eds.). Springer International Publishing, Cham, 45–50.

[41]

Nils Dahlbäck, Arne Jönsson, and Lars Ahrenberg. 1993. Wizard of Oz studies—why and how. Knowledge-based systems 6, 4 (1993), 258–266.

[42]

M. Day, M. R. Penumala, and J. Gonzalez-Sanchez. 2019. Annete: An Intelligent Tutoring Companion Embedded into the Eclipse IDE. In 2019 IEEE First International Conference on Cognitive Machine Intelligence (CogMI). IEEE, New York, US, 71–80.

[43]

Claudio León de la Barra and Broderick Crawford. 2007. Fostering Creativity Thinking in Agile Software Development. In HCI and Usability for Medicine and Health Care, Andreas Holzinger (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 415–426.

[44]

Angel de Vicente and Helen Pain. 1998. Motivation Diagnosis in Intelligent Tutoring Systems. In Intelligent Tutoring Systems, Barry P. Goettl, Henry M. Halff, Carol L. Redfield, and Valerie J. Shute (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 86–95.

[45]

Edward L Deci, Anja H Olafsen, and Richard M Ryan. 2017. Self-determination theory in work organizations: The state of a science. Annual Review of Organizational Psychology and Organizational Behavior 4 (2017), 19–43.

[46]

Doris M. Dehn and Susanne van Mulken. 2000. The Impact of Animated Interface Agents: A Review of Empirical Research. Int. J. Hum.-Comput. Stud. 52, 1 (Jan. 2000), 1–22. https://doi.org/10.1006/ijhc.1999.0325

Digital Library

[47]

Tom DeMarco and Tim Lister. 2013. Peopleware: Productive Projects and Teams (3rd Edition) (3rd ed.). Addison-Wesley Professional, Boston, MA, USA.

[48]

Melissa C. Duffy and Roger Azevedo. 2015. Motivation matters: Interactions between achievement goals and agent scaffolding for self-regulated learning within an intelligent tutoring system. Computers in Human Behavior 52 (2015), 338 – 348. http://www.sciencedirect.com/science/article/pii/S0747563215004227

Digital Library

[49]

Rafael Duque and Crescencio Bravo. 2008. Analyzing Work Productivity and Program Quality in Collaborative Programming. In Proceedings of the 2008 The Third International Conference on Software Engineering Advances(ICSEA ’08). IEEE Computer Society, Washington, DC, USA, 270–276. https://doi.org/10.1109/ICSEA.2008.82

Digital Library

[50]

Tore Dybå, Erik Arisholm, Dag Sjøberg, Jo Hannay, and Forrest Shull. 2007. Are Two Heads Better than One? On the Effectiveness of Pair Programming. Software, IEEE 24 (12 2007), 12 – 15. https://doi.org/10.1109/MS.2007.158

Digital Library

[51]

Berland Edelman and Inc. 2010. Creativity and education: Why it matters. Adobe. Retrieved September 18th, 2019 from http://www.adobe.com/aboutadobe/pressroom/pdfs/Adobe_Creativity_and_Education_Why_It_Matters_study.pdf

[52]

Katrina Falkner, Nickolas J.G. Falkner, and Rebecca Vivian. 2013. Collaborative Learning and Anxiety: A Phenomenographic Study of Collaborative Learning Activities. In Proceeding of the 44th ACM Technical Symposium on Computer Science Education (Denver, Colorado, USA) (SIGCSE ’13). Association for Computing Machinery, New York, NY, USA, 227–232. https://doi.org/10.1145/2445196.2445268

Digital Library

[53]

Carmen Fischer, Charlotte P. Malycha, and Ernestine Schafmann. 2019. The Influence of Intrinsic Motivation and Synergistic Extrinsic Motivators on Creativity and Innovation. Frontiers in Psychology 10 (2019), 137. https://doi.org/10.3389/fpsyg.2019.00137

[54]

S.T. Fiske, E.H.P.P.S.T. Fiske, and S.E. Taylor. 1991. Social Cognition. McGraw-Hill, New York City, USA. https://books.google.com/books?id=6Uq3QgAACAAJ

[55]

H. Gallis, Erik Arisholm, and Tore Dybå. 2002. A Transition From Partner Programming to Pair Programming - an Industrial Case Study. In Workshop ”Pair Programming Installed” in 17th Annual ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), Position paper. ACM, Seattle USA, –.

[56]

H. Gallis, E. Arisholm, and T. Dyba. 2003. An initial framework for research on pair programming. In 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings.ACM, NY, USA, 132–142. https://doi.org/10.1109/ISESE.2003.1237972

[57]

H. Gallis, E. Arisholm, and T. Dyba. 2003. An initial framework for research on pair programming. In 2003 International Symposium on Empirical Software Engineering. ACM, NY, USA, 132–142.

[58]

Alex Gerdes, Bastiaan Heeren, Johan Jeuring, and L. Thomas van Binsbergen. 2016. Ask-Elle: an Adaptable Programming Tutor for Haskell Giving Automated Feedback. International Journal of Artificial Intelligence in Education 27 (02 2016). https://doi.org/10.1007/s40593-015-0080-x

[59]

Nadine Glas and Catherine Pelachaud. 2015. Definitions of Engagement in Human-Agent Interaction. In International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, USA, 944–949. https://doi.org/10.1109/ACII.2015.7344688

Digital Library

[60]

Barney G. Glaser and Anselm L. Strauss. 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine de Gruyter, New York, NY.

[61]

Bradley Goodman, Amy Soller, Frank Linton, Robert Gaimari, and The Mitre. 1998. Encouraging student reflection and articulation using a learning companion. International Journal of Artificial Intelligence in Education 9 (1998), 237–255.

[62]

Google. 2019. Google text-to-speech python library. https://github.com/pndurette/gTTS

[63]

Google. 2020. Virtual Assistant Google Assistant. https://assistant.google.com/

[64]

Jonathan Gratch, Ning Wang, Jillian Gerten, Edward Fast, and Robin Duffy. 2007. Creating Rapport with Virtual Agents. In Intelligent Virtual Agents, Catherine Pelachaud, Jean-Claude Martin, Elisabeth André, Gérard Chollet, Kostas Karpouzis, and Danielle Pelé (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 125–138.

[65]

J.P. Guilford. 1968. Intelligence, Creativity, and Their Educational Implications. R. R. Knapp, Open Library. https://books.google.com/books?id=WE8kAQAAMAAJ

[66]

Keun-Woo Han, EunKyoung Lee, and Youngjun Lee. 2010. The Impact of a Peer-Learning Agent Based on Pair Programming in a Programming Course. Education, IEEE Transactions on 53 (06 2010), 318 – 327. https://doi.org/10.1109/TE.2009.2019121

Digital Library

[67]

Brian Hanks. 2008. Empirical evaluation of distributed pair programming. International Journal of Human-Computer Studies 66 (07 2008), 530–544. https://doi.org/10.1016/j.ijhcs.2007.10.003

Digital Library

[68]

Brian F. Hanks. 2004. Distributed Pair Programming: An Empirical Study. In Extreme Programming and Agile Methods - XP/Agile Universe 2004, Carmen Zannier, Hakan Erdogmus, and Lowell Lindstrom(Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 81–91.

[69]

Dai Hasegawa, Justine Cassell, and Kenji Araki. 2010. The Role of Embodiment and Perspective in Direction-Giving Systems. In Proceedings of AAAI Fall Workshop on Dialog with Robots. AAAI PRESS, USA.

[70]

Renate Häuslschmid, Max von Bülow, Bastian Pfleging, and Andreas Butz. 2017. SupportingTrust in Autonomous Driving. In Proceedings of the 22Nd International Conference on Intelligent User Interfaces (Limassol, Cyprus) (IUI ’17). ACM, New York, NY, USA, 319–329. https://doi.org/10.1145/3025171.3025198

Digital Library

[71]

Charles G. Hill, Maren Haag, Alannah Oleson, Chris Mendez, Nicola Marsden, Anita Sarma, and Margaret Burnett. 2017. Gender-Inclusiveness Personas vs. Stereotyping: Can We Have It Both Ways?. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 6658–6671. https://doi.org/10.1145/3025453.3025609

Digital Library

[72]

Anthony J. Hillesheim, Christina F. Rusnock, Jason M. Bindewald, and Michael E. Miller. 2017. Relationships between User Demographics and User Trust in an Autonomous Agent. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 61, 1(2017), 314–318. https://doi.org/10.1177/1541931213601560

[73]

Kevin Anthony Hoff and Masooda Bashir. 2015. Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust. Human Factors 57, 3 (2015), 407–434. https://doi.org/10.1177/0018720814547570PMID: 25875432.

[74]

IBM. 2019. Eclipse IDE. https://www.eclipse.org/

[75]

Paul Jaccard. 1901. Etude de la distribution florale dans une portion des Alpes et du Jura. Bulletin de la Societe Vaudoise des Sciences Naturelles 37 (01 1901), 547–579. https://doi.org/10.5169/seals-266450

[76]

Mohit Jain, Pratyush Kumar, Ishita Bhansali, Q. Vera Liao, Khai Truong, and Shwetak Patel. 2018. FarmChat: A Conversational Agent to Answer Farmer Queries. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 4, Article 170 (Dec. 2018), 22 pages. https://doi.org/10.1145/3287048

Digital Library

[77]

Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and Informing the Design of Chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 895–906. https://doi.org/10.1145/3196709.3196735

Digital Library

[78]

D. James and S. Clarke. 1998. Women, men, and interruptions: A critical review. In D. Tannen (Ed.), Oxford studies in sociolinguistics. Gender and conversational interaction. Oxford University Press, UK, 231–280.

[79]

Lindsay Jarratt, Nicholas A. Bowman, K.C. Culver, and Alberto Maria Segre. 2019. A Large-Scale Experimental Study of Gender and Pair Composition in Pair Programming. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (Aberdeen, Scotland Uk) (ITiCSE ’19). Association for Computing Machinery, New York, NY, USA, 176–181. https://doi.org/10.1145/3304221.3319782

Digital Library

[80]

Will Jerigan, Amber Horvath, Michael Lee, Margaret Burnett, Cuilty Taylor, Sandeep Kuttal, Anicia Peters, Irwin Kwan, Faezeh Bahmani, and Andrew Ko. 2015. A Principled Evaluation for a Principled Idea Garden. In 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, USA. https://doi.org/10.1109/VLHCC.2015.7357222

[81]

William Jernigan, Amber Horvath, Michael Lee, Margaret Burnett, Taylor Cuilty, Sandeep Kuttal, Anicia Peters, Irwin Kwan, Faezeh Bahmani, Andrew Ko, Christopher J. Mendez, and Alannah Oleson. 2017. General principles for a Generalized Idea. Journal of Visual Languages & Computing 39 (2017), 51 – 65. https://doi.org/10.1016/j.jvlc.2017.04.005Special Issue on Programming and Modelling Tools.

Digital Library

[82]

Xiaoming Jiang and Marc D. Pell. 2015. On how the brain decodes vocal cues about speaker confidence. Cortex 66(2015), 9 – 34. http://www.sciencedirect.com/science/article/pii/S0010945215000593

[83]

Ewa Kacewicz, James W. Pennebaker, Matthew Davis, Moongee Jeon, and Arthur C. Graesser. 2014. Pronoun Use Reflects Standings in Social Hierarchies. Journal of Language and Social Psychology 33, 2 (2014), 125–143. https://doi.org/10.1177/0261927X13502654

[84]

Peter H. Kahn, Nathan G. Freier, Takayuki Kanda, Hiroshi Ishiguro, Jolina H. Ruckert, Rachel L. Severson, and Shaun K. Kane. 2008. Design Patterns for Sociality in Human-Robot Interaction. In Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction (Amsterdam, The Netherlands) (HRI ’08). Association for Computing Machinery, New York, NY, USA, 97–104. https://doi.org/10.1145/1349822.1349836

Digital Library

[85]

Alexander Karan, Robert Rosenthal, and Megan L. Robbins. 2019. Meta-analytic evidence that we-talk predicts relationship and personal functioning in romantic couples. Journal of Social and Personal Relationships 36, 9(2019), 2624–2651. https://doi.org/10.1177/0265407518795336

[86]

Neha Katira, Laurie Williams, Eric Wiebe, Carol Miller, Suzanne Balik, and Ed Gehringer. 2004. On Understanding Compatibility of Student Pair Programmers. In Proceedings of the 35th SIGCSE Technical Symposium on Computer Science Education (Norfolk, Virginia, USA) (SIGCSE ’04). Association for Computing Machinery, New York, NY, USA, 7–11. https://doi.org/10.1145/971300.971307

Digital Library

[87]

S. Kaur Kuttal, K. Gerstner, and A. Bejarano. 2019. Remote Pair Programming in Online CS Education: Investigating through a Gender Lens. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, USA, 75–85.

[88]

Iman Keivanloo, Juergen Rilling, and Ying Zou. 2014. Spotting Working Code Examples. In Proceedings of the 36th International Conference on Software Engineering (Hyderabad, India) (ICSE 2014). Association for Computing Machinery, New York, NY, USA, 664–675. https://doi.org/10.1145/2568225.2568292

Digital Library

[89]

Kisub Kim, Dongsun Kim, Tegawendé F. Bissyandé, Eunjong Choi, Li Li, Jacques Klein, and Yves Le Traon. 2018. FaCoY: A Code-to-Code Search Engine. In Proceedings of the 40th International Conference on Software Engineering (Gothenburg, Sweden) (ICSE ’18). Association for Computing Machinery, New York, NY, USA, 946–957. https://doi.org/10.1145/3180155.3180187

Digital Library

[90]

Barbara Kitchenham, S.L. Pfleeger, L.M. Pickard, Peter Jones, David Hoaglin, Khaled Emam, and Jarrett Rosenberg. 2002. Preliminary Guidelines for Empirical Research in Software Engineering. Software Engineering, IEEE Transactions on 28 (09 2002), 721– 734. https://doi.org/10.1109/TSE.2002.1027796

Digital Library

[91]

Dominique Knutsen and Ludovic Le Bigot. 2014. The Influence of Reference Acceptance and Reuse on Conversational Memory Traces. Journal of experimental psychology. Learning, memory, and cognition 41 (07 2014). https://doi.org/10.1037/xlm0000036

[92]

Dominique Knutsen, Ludovic Le Bigot, and Christine Ros. 2017. Explicit feedback from users attenuates memory biases in human-system dialogue. International Journal of Human-Computer Studies 97 (2017), 77 – 87. http://www.sciencedirect.com/science/article/pii/S1071581916301045

[93]

Dominique Knutsen, Christine Ros, and Ludovic Le Bigot. 2016. Generating References in Naturalistic Face-to-Face and Phone-Mediated Dialog Settings. Topics in Cognitive Science 8 (08 2016). https://doi.org/10.1111/tops.12218

[94]

Andrew J. Ko and Brad A. Myers. 2004. Designing the Whyline: A Debugging Interface for Asking Questions about Program Behavior. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI ’04). Association for Computing Machinery, New York, NY, USA, 151–158. https://doi.org/10.1145/985692.985712

Digital Library

[95]

Andrew J. Ko and Brad A. Myers. 2008. Debugging Reinvented: Asking and Answering Why and Why Not Questions about Program Behavior. In Proceedings of the 30th International Conference on Software Engineering (Leipzig, Germany) (ICSE ’08). Association for Computing Machinery, New York, NY, USA, 301–310. https://doi.org/10.1145/1368088.1368130

Digital Library

[96]

Kuttal, Kwasny, Ong, and Robe. 2020. Correctness and Progress Metrics. https://docs.google.com/spreadsheets/d/1UawMLVACqLjC6JH7I5vwWVQLCv6ph-lkCllZDwHkM3Y/edit?usp=sharing

[97]

S. K. Kuttal, J. Myers, S. Gurka, D. Magar, D. Piorkowski, and R. Bellamy. 2020. Towards Designing Conversational Agents for Pair Programming: Accounting for Creativity Strategies and Conversational Styles. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, USA, 1–11.

[98]

Danielle L. Jones and Scott D. Fleming. 2013. What use is a backseat driver? A qualitative investigation of pair programming. In Proceedings of IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC. IEEE, USA, 103–110.

[99]

Thomas K Landauer. 1987. Psychology as a mother of invention. ACM SIGCHI Bulletin 18, 4 (1987), 333–335.

Digital Library

[100]

Susan Leavy. 2018. Gender Bias in Artificial Intelligence: The Need for Diversity and Gender Theory in Machine Learning. In Proceedings of the 1st International Workshop on Gender Equality in Software Engineering (Gothenburg, Sweden) (GE ’18). Association for Computing Machinery, New York, NY, USA, 14–16. https://doi.org/10.1145/3195570.3195580

Digital Library

[101]

Susan Leavy. 2018. Gender Bias in Artificial Intelligence: The Need for Diversity and Gender Theory in Machine Learning. In Proceedings of the 1st International Workshop on Gender Equality in Software Engineering (Gothenburg, Sweden) (GE ’18). Association for Computing Machinery, New York, NY, USA, 14–16. https://doi.org/10.1145/3195570.3195580

Digital Library

[102]

Marvin Levine. 1988. Effective problem solving. Prentice Hall, NJ, USA.

[103]

Clayton Lewis. 1982. Using the ”thinking-aloud” method in cognitive interface design. IBM T.J. Watson Research Center, Yorktown Heights, N.Y.

[104]

Colleen M. Lewis and Niral Shah. 2015. How Equity and Inequity Can Emerge in Pair Programming. In Proceedings of the Eleventh Annual International Conference on International Computing Education Research(Omaha, Nebraska, USA) (ICER ’15). Association for Computing Machinery, New York, NY, USA, 41–50. https://doi.org/10.1145/2787622.2787716

Digital Library

[105]

Zhen Li and Eileen Kraemer. 2014. Social Effects of Pair Programming Mitigate Impact of Bounded Rationality. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (Atlanta, Georgia, USA) (SIGCSE ’14). Association for Computing Machinery, New York, NY, USA, 385–390. https://doi.org/10.1145/2538862.2538968

Digital Library

[106]

Dapeng Liu, Andrian Marcus, Denys Poshyvanyk, and Vaclav Rajlich. 2007. Feature Location via Information Retrieval Based Filtering of a Single Scenario Execution Trace. In Proceedings of the Twenty-Second IEEE/ACM International Conference on Automated Software Engineering (Atlanta, Georgia, USA) (ASE ’07). Association for Computing Machinery, New York, NY, USA, 234–243. https://doi.org/10.1145/1321631.1321667

Digital Library

[107]

Zhiqiang Liu and Dieter J Schonwetter. 2004. Teaching creativity in engineering. International Journal of Engineering Education 20, 5 (2004), 801–808.

[108]

Irene Lopatovska and Harriet Williams. 2018. Personification of the Amazon Alexa: BFF or a Mindless Companion. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (New Brunswick, NJ, USA) (CHIIR ’18). Association for Computing Machinery, New York, NY, USA, 265–268. https://doi.org/10.1145/3176349.3176868

Digital Library

[109]

Ewa Luger and Abigail Sellen. 2016. ”Like Having a Really Bad PA”: The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 5286–5297. https://doi.org/10.1145/2858036.2858288

Digital Library

[110]

A. Marcus, V. Rajlich, J. Buchta, M. Petrenko, and A. Sergeyev. 2005. Static techniques for concept location in object-oriented code. In 13th International Workshop on Program Comprehension (IWPC’05). IEEE Computer Society, Los Alamitos, CA, USA, 33–42.

[111]

Yukihiro Matsubara and Mitsuo Nagamachi. 1996. Motivation system and human model for intelligent tutoring. In Intelligent Tutoring Systems, Claude Frasson, Gilles Gauthier, and Alan Lesgold (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 139–147.

[112]

Charlie McDowell, Linda Werner, Heather Bullock, and Julian Fernald. 2002. The Effects of Pair-Programming on Performance in an Introductory Programming Course. In Proceedings of the 33rd SIGCSE Technical Symposium on Computer Science Education(Cincinnati, Kentucky) (SIGCSE ’02). Association for Computing Machinery, New York, NY, USA, 38–42. https://doi.org/10.1145/563340.563353

Digital Library

[113]

Charles Mcdowell, Linda Werner, Heather Bullock, and Julian Fernald. 2006. Pair programming improves student retention, confidence, and program quality. Commun. ACM 49 (08 2006), 90–95. https://doi.org/10.1145/1145293

[114]

Charlie McDowell, Linda Werner, Heather E. Bullock, and Julian Fernald. 2003. The Impact of Pair Programming on Student Performance, Perception and Persistence. In Proceedings of the 25th International Conference on Software Engineering (Portland, Oregon) (ICSE ’03). IEEE Computer Society, USA, 602–607.

Digital Library

[115]

Meiliana, Irwandhi Septian, Ricky Setiawan Alianto, Daniel, and Ford Lumban Gaol. 2017. Automated Test Case Generation from UML Activity Diagram and Sequence Diagram using Depth First Search Algorithm. Procedia Computer Science 116 (2017), 629 – 637. http://www.sciencedirect.com/science/article/pii/S1877050917320732 Discovery and innovation of computer science technology in artificial intelligence era: The 2nd International Conference on Computer Science and Computational Intelligence (ICCSCI 2017).

Digital Library

[116]

Christopher Mendez, Hema Susmita Padala, Zoe Steine-Hanson, Claudia Hilderbrand, Amber Horvath, Charles Hill, Logan Simpson, Nupoor Patil, Anita Sarma, and Margaret Burnett. 2018. Open Source Barriers to Entry, Revisited: A Sociotechnical Perspective. In Proceedings of the 40th International Conference on Software Engineering (Gothenburg, Sweden) (ICSE ’18). Association for Computing Machinery, New York, NY, USA, 1004–1015. https://doi.org/10.1145/3180155.3180241

Digital Library

[117]

M. Minsky, R. Kurzweil, and S. Mann. 2013. The society of intelligent veillance. In 2013 IEEE International Symposium on Technology and Society (ISTAS): Social Implications of Wearable Computing and Augmediated Reality in Everyday Life. IEEE, USA, 13–17.

[118]

Matheus Monteiro, Erica Souza, Andre Endo, and Nandamudi Vijaykumar. 2019. Analyzing graph-based algorithms employed to generate test cases from finite state machines. In 2019 IEEE Latin American Test Symposium (LATS). IEEE, USA. https://doi.org/10.1109/LATW.2019.8704603

[119]

Susanne van Mulken, Elisabeth André, and Jochen Müller. 1999. An Empirical Study on the Trustworthiness of Life-like Interface Agents. In Proceedings of the HCI International ’99 (the 8th International Conference on Human-Computer Interaction) on Human-Computer Interaction: Communication, Cooperation, and Application Design-Volume 2 - Volume 2. L. Erlbaum Associates Inc., Hillsdale, NJ, USA, 152–156.

[120]

Kumiko Murata. 1994. Intrusive or co-operative? A cross-cultural study of interruption. Journal of Pragmatics 21, 4 (1994), 385 – 400. http://www.sciencedirect.com/science/article/pii/0378216694900116

[121]

Emerson Murphy-Hill and Gail C. Murphy. 2011. Peer Interaction Effectively, yet Infrequently, Enables Programmers to Discover New Tools. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work (Hangzhou, China) (CSCW ’11). Association for Computing Machinery, New York, NY, USA, 405–414. https://doi.org/10.1145/1958824.1958888

Digital Library

[122]

Nachiappan Nagappan, Laurie Williams, Miriam Ferzli, Eric Wiebe, Kai Yang, Carol Miller, and Suzanne Balik. 2003. Improving the CS1 Experience with Pair Programming. SIGCSE Bull. 35, 1 (Jan. 2003), 359–362. https://doi.org/10.1145/792548.612006

Digital Library

[123]

Sik Hung Ng, Mark Brooke, and Michael Dunne. 1995. Interruption and Influence in Discussion Groups. Journal of Language and Social Psychology 14, 4 (1995), 369–381. https://doi.org/10.1177/0261927X950144003

[124]

Haoran Niu, Iman Keivanloo, and Ying Zou. 2017. Learning to rank code examples for code search engines. Empirical Software Engineering 22, 1 (2017), 259–291.

Digital Library

[125]

John Nosek. 1998. The Case for Collaborative Programming. Commun. ACM 41 (03 1998). https://doi.org/10.1145/272287.272333

Digital Library

[126]

Clem O’Donnell, Jim Buckley, Abdulhussain Mahdi, John Nelson, and Michael English. 2015. Evaluating Pair-Programming for Non-Computer Science Major Students. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (Kansas City, Missouri, USA) (SIGCSE ’15). Association for Computing Machinery, New York, NY, USA, 569–574. https://doi.org/10.1145/2676723.2677289

Digital Library

[127]

Sharon Oviatt and Philip Cohen. 2000. Perceptual User Interfaces: Multimodal Interfaces That Process What Comes Naturally. Commun. ACM 43, 3 (March 2000), 45–53. https://doi.org/10.1145/330534.330538

Digital Library

[128]

Frank Pajares. 2002. Overview of social cognitive theory and of self-efficacy.

[129]

David Walsh Palmieri. 2002. Knowledge Management Through Pair Programming.

[130]

George Polya. 2004. How to solve it: A new aspect of mathematical method. Vol. 85. Princeton university press, NJ, USA.

[131]

Leo Porter and Beth Simon. 2013. Retaining Nearly One-Third More Majors with a Trio of Instructional Best Practices in CS1. In Proceeding of the 44th ACM Technical Symposium on Computer Science Education (Denver, Colorado, USA) (SIGCSE ’13). Association for Computing Machinery, New York, NY, USA, 165–170. https://doi.org/10.1145/2445196.2445248

Digital Library

[132]

Lutz Prechelt, Ulrich Stärk, and Stephan Salinger. 2009. 7 Types of Cooperation Episodes in Side-by-Side Programming. In Proc. 21st Annual Workshop of the Psychology of Programming Interest Group (PPIG ’09). ACM, USA.

[133]

Mukund Raghothaman, Yi Wei, and Youssef Hamadi. 2016. SWIM: Synthesizing What i Mean: Code Search and Idiomatic Snippet Synthesis. In Proceedings of the 38th International Conference on Software Engineering (Austin, Texas) (ICSE ’16). Association for Computing Machinery, New York, NY, USA, 357–367. https://doi.org/10.1145/2884781.2884808

Digital Library

[134]

P. Rane. 2017. Automatic Generation of Test Cases for Agile using Natural Language Processing.

[135]

Genaro Rebolledo-Mendez, Sara de Freitas, Jose Rafael Rojano-Caceres, and Alma Rosa García-Gaona. 2010. An Empirical Examination of the Relation Between Attention and Motivation in Computer-Based Education: a Modeling Approach. In Proceedings of the Twenty-Third International Florida Artificial Intelligence Research Society Conference, May 19-21, 2010, Daytona Beach, Florida, USA, Hans W. Guesgen and R. Charles Murray (Eds.). AAAI Press, USA, 74–79.

[136]

Genaro Rebolledo-Mendez, Benedict du Boulay, and Rosemary Luckin. 2006. Motivating the Learner: An Empirical Evaluation. In Intelligent Tutoring Systems, Mitsuru Ikeda, Kevin D. Ashley, and Tak-Wai Chan (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 545–554.

[137]

Laurel D Riek. 2012. Wizard of oz studies in hri: a systematic review and new reporting guidelines. Journal of Human-Robot Interaction 1, 1 (2012), 119–136.

Digital Library

[138]

Peter Robe. 2020. Designing PairBuddy – Conversational Agent for Pair. https://drive.google.com/drive/folders/1vIOdro0pg8C1jSB42KzYrDRKO0PVhqZ1?usp=sharing

[139]

T. J. Robertson, Shrinu Prabhakararao, Margaret Burnett, Curtis Cook, Joseph R. Ruthruff, Laura Beckwith, and Amit Phalgune. 2004. Impact of Interruption Style on End-User Debugging. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI ’04). Association for Computing Machinery, New York, NY, USA, 287–294. https://doi.org/10.1145/985692.985729

Digital Library

[140]

Fernando J. Rodríguez, Kimberly Michelle Price, and Kristy Elizabeth Boyer. 2017. Exploring the Pair Programming Process: Characteristics of Effective Collaboration. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (Seattle, Washington, USA) (SIGCSE ’17). Association for Computing Machinery, New York, NY, USA, 507–512. https://doi.org/10.1145/3017680.3017748

Digital Library

[141]

Omar Ruvalcaba, Linda Werner, and Jill Denner. 2016. Observations of Pair Programming: Variations in Collaboration Across Demographic Groups. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (Memphis, Tennessee, USA) (SIGCSE ’16). Association for Computing Machinery, New York, NY, USA, 90–95. https://doi.org/10.1145/2839509.2844558

Digital Library

[142]

Jeffrey S. Saltz and Ivan Shamshurin. 2019. Exploring pair programming beyond computer science: a case study in its use in data science/data engineering. International Journal of Higher Education and Sustainability 2, 4(2019), 265–278. https://doi.org/10.1504/IJHES.2019.103360

[143]

T. Savage, M. Revelle, and D. Poshyvanyk. 2010. FLAT3: feature location and textual tracing tool. In 2010 ACM/IEEE 32nd International Conference on Software Engineering, Vol. 2. ACM, USA, 255–258.

[144]

Philippe G. Schyns, Lucy S. Petro, and Marie L. Smith. 2009. Transmission of Facial Expressions of Emotion Co-Evolved with Their Efficient Decoding in the Brain: Behavioral and Brain Evidence. PLOS ONE 4, 5 (05 2009), 1–16. https://doi.org/10.1371/journal.pone.0005625

[145]

C. B. Seaman. 1999. ”Qualitative Methods in Empirical Studies of Software Engineering”. IEEE Transactions on Software Engineering 25, 4 (1999), 557–572.

Digital Library

[146]

A. Seeger, J. Pfeiffer, and A. Heinzl. 2017. When Do We Need a Human? Anthropomorphic Design and Trustworthiness of Conversational Agents. In Proceedings of the SixteenthAnnual Pre-ICIS Workshop on HCI Research in MIS. ACM, USA, 1–5.

[147]

Ameneh Shamekhi, Q. Vera Liao, Dakuo Wang, Rachel K. E. Bellamy, and Thomas Erickson. 2018. Face Value? Exploring the Effects of Embodiment for a Group Facilitation Agent. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173965

Digital Library

[148]

R. Sharma, S. Gulia, and K. K. Biswas. 2014. Automated generation of activity and sequence diagrams from natural language requirements. In 9th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE). SCITEPRESS Digital Library, Portugal, 1–9.

[149]

Arun Shekhar and Nicola Marsden. 2018. Cognitive Walkthrough of a Learning Management System with Gendered Personas. In Proceedings of the 4th Conference on Gender & IT (Heilbronn, Germany) (GenderIT ’18). Association for Computing Machinery, New York, NY, USA, 191–198. https://doi.org/10.1145/3196839.3196869

Digital Library

[150]

F. Shull, J. Singer, and D. I. K. Sjøberg. 2008. Guide to Advanced Empirical Software Engineering: Springer.

[151]

Candace L. Sidner, Christopher Lee, Cory D. Kidd, Neal Lesh, and Charles Rich. 2005. Explorations in engagement for humans and robots. Artificial Intelligence 166, 1 (2005), 140 – 164. http://www.sciencedirect.com/science/article/pii/S0004370205000512

[152]

Simon and Susan Snowdon. 2011. Explaining Program Code: Giving Students the Answer Helps - but Only Just. In Proceedings of the Seventh International Workshop on Computing Education Research (Providence, Rhode Island, USA) (ICER ’11). Association for Computing Machinery, New York, NY, USA, 93–100. https://doi.org/10.1145/2016911.2016931

Digital Library

[153]

Zach Sims. 2020. Code Academy. https://www.codecademy.com/

[154]

Arsenij Solovjev. 2020. Saros Project. https://www.saros-project.org/

[155]

Jörg Spieler. 2020. UCDetector. Open Source. http://www.ucdetector.org/

[156]

Hugo Spiers, Bradley Love, Mike Pelley, Charlotte Gibb, and Robin Murphy. 2016. Anterior Temporal Lobe Tracks the Formation of Prejudice. Journal of Cognitive Neuroscience 29 (10 2016), 1–15. https://doi.org/10.1162/jocn_a_01056

Digital Library

[157]

Lee Sproull, Mani Subramani, Sara Kiesler, Janet H. Walker, and Keith Waters. 1996. When the Interface is a Face. Hum.-Comput. Interact. 11, 2 (June 1996), 97–124. https://doi.org/10.1207/s15327051hci1102_1

Digital Library

[158]

R. Strachan, A. Peixoto, I. Emembolu, and M. T. Restivo. 2018. Women in engineering: Addressing the gender gap, exploring trust and our unconscious bias. In 2018 IEEE Global Engineering Education Conference (EDUCON). IEEE, USA, 2088–2093.

[159]

Anselm L. Strauss and Juliet M. Corbin. 1998. Basics of qualitative research: techniques and procedures for developing grounded theory. Sage Publications, Thousand Oaks, California, USA. XIII, 312 s pages.

[160]

Holotech Studios. 2020. Facerig. https://facerig.com/

[161]

S. Shyam Sundar and Jinyoung Kim. 2019. Machine Heuristic: When We Trust Computers More than Humans with Our Personal Information. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3290605.3300768

Digital Library

[162]

Akikazu Takeuchi and Taketo Naito. 1995. Situated Facial Displays: Towards Social Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’95). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 450–455. https://doi.org/10.1145/223904.223965

Digital Library

[163]

Germany TeamViewer AG. 2019. TeamViewer. https://www.teamviewer.com/en-us/

[164]

TechSmith. 2019. Morae. http://www.techsmith.com/morae.asp

[165]

Diana-Cezara Toader, Grațiela Boca, Rita Toader, Mara Măcelaru, Cezar Toader, Diana Ighian, and Adrian T. Rădulescu. 2019. The Effect of Social Presence and Chatbot Errors on Trust. Sustainability 12, 1 (Dec 2019), 256. https://doi.org/10.3390/su12010256

[166]

E. Paul Torrance. 1970. Influence of Dyadic Interaction on Creative Functioning. Psychological Reports 26, 2 (1970), 391–394. https://doi.org/10.2466/pr0.1970.26.2.391

[167]

Despina Tsompanoudi, Maya Satratzemi, Stelios Xinogalos, and Leonidas Karamitopoulos. 2019. An Empirical Study on Factors related to Distributed Pair Programming. (04 2019). https://www.learntechlib.org/p/208576

[168]

Mengping Tsuei. 2017. Learning behaviours of low-achieving children’s mathematics learning in using of helping tools in a synchronous peer-tutoring system. Interactive Learning Environments 25, 2 (2017), 147–161. https://doi.org/10.1080/10494820.2016.1276078

[169]

ALINA Tugend. 2007. Why is asking for help so difficult.

[170]

Susanne van Mulken, Elisabeth André, and Jochen Müller. 1998. The Persona Effect: How Substantial Is It?. In People and Computers XIII, Hilary Johnson, Lawrence Nigay, and Christopher Roast (Eds.). Springer London, London, 53–66.

[171]

Mihaela Vorvoreanu, Lingyi Zhang, Yun-Han Huang, Claudia Hilderbrand, Zoe Steine-Hanson, and Margaret Burnett. 2019. From Gender Biases to Gender-Inclusive Design: An Empirical Investigation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). ACM, New York, NY, USA, Article 53, 14 pages. https://doi.org/10.1145/3290605.3300283

Digital Library

[172]

Tony Wagner and Robert A Compton. 2012. Creating innovators: The making of young people who will change the world. Simon and Schuster, USA.

[173]

P. Wargnier, G. Carletti, Y. Laurent-Corniquet, S. Benveniste, P. Jouvelot, and A. Rigaud. 2016. Field evaluation with cognitively-impaired older adults of attention management in the Embodied Conversational Agent Louise. In 2016 IEEE International Conference on Serious Games and Applications for Health (SeGAH). IEEE Computer Society, Los Alamitos, CA, USA, 1–8. https://doi.org/10.1109/SeGAH.2016.7586282

[174]

Katharina Weitz, Dominik Schiller, Ruben Schlagowski, Tobias Huber, and Elisabeth André. 2019. ”Do You Trust Me?”: Increasing User-Trust by Integrating Virtual Agents in Explainable AI Interaction Design. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents (Paris, France) (IVA ’19). Association for Computing Machinery, New York, NY, USA, 7–9. https://doi.org/10.1145/3308532.3329441

Digital Library

[175]

Linda L. Werner, Brian Hanks, and Charlie McDowell. 2004. Pair-Programming Helps Female Computer Science Students. J. Educ. Resour. Comput. 4, 1 (March 2004), 4–es. https://doi.org/10.1145/1060071.1060075

Digital Library

[176]

CANDACE WEST and DON H. ZIMMERMAN. 1987. Doing Gender. Gender & Society 1, 2 (1987), 125–151. https://doi.org/10.1177/0891243287001002002

[177]

Jason Weston, Antoine Bordes, Sumit Chopra, Alexander Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov. 2015. Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks. arxiv:1502.05698 [cs.AI]

[178]

Wayne A Wickelgren. 1974. How to solve problems: Elements of a theory of problems and problem solving. WH Freeman San Francisco, USA.

[179]

Laurie Williams and Bob Kessler. 2000. The Effects of ”Pair-Pressure” and ”Pair-Learning” on Software Engineering Education. In Proceedings of the 13th Conference on Software Engineering Education & Training(CSEET ’00). IEEE Computer Society, USA, 59.

[180]

Laurie Williams and Robert Kessler. 2002. Pair Programming Illuminated. Addison-Wesley Longman Publishing Co., Inc., USA.

Digital Library

[181]

L. Williams, R. R. Kessler, W. Cunningham, and R. Jeffries. 2000. Strengthening the case for pair programming. IEEE Software 17, 4 (2000), 19–25.

Digital Library

[182]

Laurie Williams, D. Scott McCrickard, Lucas Layman, and Khaled Hussein. 2008. Eleven Guidelines for Implementing Pair Programming in the Classroom. In Proceedings of the Agile 2008(AGILE ’08). IEEE Computer Society, USA, 445–452. https://doi.org/10.1109/Agile.2008.12

Digital Library

[183]

Laurie Williams, Charlie McDowell, Nachiappan Nagappan, Julian Fernald, and Linda Werner. 2003. Building Pair Programming Knowledge through a Family of Experiments. In Proceedings of the 2003 International Symposium on Empirical Software Engineering(ISESE ’03). IEEE Computer Society, USA, 143.

Digital Library

[184]

Laurie Williams, Eric Wiebe, Kai Yang, Miriam Ferzli, and Carol Miller. 2002. In Support of Pair Programming in the Introductory Computer Science Course. Computer Science Education 12, 3 (2002), 197–212. https://doi.org/10.1076/csed.12.3.197.8618

[185]

Laurie A. Williams. 2010. Pair Programming. John Wiley & Sons, USA. 311–322 pages.

[186]

Laurie A. Williams and Robert R. Kessler. 2000. All I Really Need to Know about Pair Programming I Learned in Kindergarten. Commun. ACM 43, 5 (May 2000), 108–114. https://doi.org/10.1145/332833.332848

Digital Library

[187]

Aaron Wilson, Margaret Burnett, Laura Beckwith, Orion Granatir, Ledah Casburn, Curtis Cook, Mike Durham, and Gregg Rothermel. 2003. Harnessing Curiosity to Increase Correctness in End-User Programming. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI ’03). Association for Computing Machinery, New York, NY, USA, 305–312. https://doi.org/10.1145/642611.642665

Digital Library

[188]

James Wilson and Daniel Rosenberg. 1988. Rapid prototyping for user interface design. In Handbook of human-computer interaction. Elsevier, Amsterdam, Netherlands, 859–875.

[189]

Anita Woolley, Ishani Aggarwal, and Thomas Malone. 2015. Collective Intelligence and Group Performance. Current Directions in Psychological Science 24 (12 2015), 420–424. https://doi.org/10.1177/0963721415599543

[190]

Nick Yee, Jeremy N Bailenson, and Kathryn Rickertsen. 2007. A Meta-analysis of the Impact of the Inclusion and Realism of Human-like Faces on User Experiences in Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). ACM, New York, NY, USA, 1–10. https://doi.org/10.1145/1240624.1240626

Digital Library

[191]

Kimberly Michelle Ying, Lydia G. Pezzullo, Mohona Ahmed, Kassandra Crompton, Jeremiah Blanchard, and Kristy Elizabeth Boyer. 2019. In Their Own Words: Gender Differences in Student Perceptions of Pair Programming. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education(Minneapolis, MN, USA) (SIGCSE ’19). Association for Computing Machinery, New York, NY, USA, 1053–1059. https://doi.org/10.1145/3287324.3287380

Digital Library

[192]

Jeff Youngquist. 2009. The Effect of Interruptions and Dyad Gender Combination on Perceptions of Interpersonal Dominance. Communication Studies 60, 2 (2009), 147–163. https://doi.org/10.1080/10510970902834874

[193]

Hans Yuan and Yingjun Cao. 2019. Hybrid Pair Programming - A Promising Alternative to Standard Pair Programming. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education(Minneapolis, MN, USA) (SIGCSE ’19). Association for Computing Machinery, New York, NY, USA, 1046–1052. https://doi.org/10.1145/3287324.3287352

Digital Library

[194]

Mohan Zalake, Julia Woodward, Amanpreet Kapoor, and Benjamin Lok. 2018. Assessing the Impact of Virtual Human’s Appearance on Users’ Trust Levels. In Proceedings of the 18th International Conference on Intelligent Virtual Agents (Sydney, NSW, Australia) (IVA ’18). Association for Computing Machinery, New York, NY, USA, 329–330. https://doi.org/10.1145/3267851.3267863

Digital Library

[195]

Yong Zhao. 2012. World class learners: Educating creative and entrepreneurial students. Corwin Press, USA.

[196]

Rui Zhi, Samiha Marwan, Yihuan Dong, Nicholas Lytle, Thomas W Price, and Tiffany Barnes. 2019. Toward Data-Driven Example Feedback for Novice Programming. In International Educational Data Mining Society. ERIC, USA, 218–227.

[197]

Franz Zieris and Lutz Prechelt. 2014. On Knowledge Transfer Skill in Pair Programming. In Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (Torino, Italy) (ESEM ’14). Association for Computing Machinery, New York, NY, USA, Article 11, 10 pages. https://doi.org/10.1145/2652524.2652529

Digital Library

[198]

Franz Zieris and Lutz Prechelt. 2020. Explaining Pair Programming Session Dynamics from Knowledge Gaps. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 421–432. https://doi.org/10.1145/3377811.3380925

Digital Library

Cited By

Hou XWu ZWang XEricson BJoyner DKim MWang XXia M(2024)CodeTailor: LLM-Powered Personalized Parsons Puzzles for Engaging Support While Learning ProgrammingProceedings of the Eleventh ACM Conference on Learning @ Scale10.1145/3657604.3662032(51-62)Online publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1145/3657604.3662032
Naik AYin JKamath AMa QWu SMurray CBogart CSakr MRose C(2024)Generating Situated Reflection Triggers About Alternative Solution Paths: A Case Study of Generative AI for Computer-Supported Collaborative LearningArtificial Intelligence in Education10.1007/978-3-031-64302-6_4(46-59)Online publication date: 2-Jul-2024
https://doi.org/10.1007/978-3-031-64302-6_4
Watanabe KDengel AIshimaru S(2023)Accelerating Knowledge Transfer by Sensing and Actuating Social-Cognitive StatesAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610769(258-262)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1145/3594739.3610769
Show More Cited By

Index Terms

Trade-offs for Substituting a Human with an Agent in a Pair Programming Context: The Good, the Bad, and the Ugly
1. Computing methodologies
  1. Artificial intelligence
2. Human-centered computing

Index terms have been assigned to the content through auto-classification.

Recommendations

Designing PairBuddy—A Conversational Agent for Pair Programming
From automated customer support to virtual assistants, conversational agents have transformed everyday interactions, yet despite phenomenal progress, no agent exists for programming tasks. To understand the design space of such an agent, we prototyped ...
First year students' impressions of pair programming in CS1
ICER '07: Proceedings of the third international workshop on Computing education research

Pair programming, as part of the Agile Development process, has noted benefits in professional software development scenarios. These successes have led to a rise in use of pair programming in educational settings, particularly in CS1. Specifically [6] ...
First-year students' impressions of pair programming in CS1

Pair programming, as part of the Agile Development process, has noted benefits in professional software development scenarios. These successes have led to a rise in use of pair programming in educational settings, particularly in Computer Science 1 (CS1)...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

10862 pages

ISBN:9781450380966

DOI:10.1145/3411764

General Chairs:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Program Chairs:
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan
,
Publications Chairs:
Pernille Bjørn
University of Copenhagen, Denmark
,
Steven Drucker
Microsoft Research, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
890
Total Downloads

Downloads (Last 12 months)169
Downloads (Last 6 weeks)8

Reflects downloads up to 14 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hou XWu ZWang XEricson BJoyner DKim MWang XXia M(2024)CodeTailor: LLM-Powered Personalized Parsons Puzzles for Engaging Support While Learning ProgrammingProceedings of the Eleventh ACM Conference on Learning @ Scale10.1145/3657604.3662032(51-62)Online publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1145/3657604.3662032
Naik AYin JKamath AMa QWu SMurray CBogart CSakr MRose C(2024)Generating Situated Reflection Triggers About Alternative Solution Paths: A Case Study of Generative AI for Computer-Supported Collaborative LearningArtificial Intelligence in Education10.1007/978-3-031-64302-6_4(46-59)Online publication date: 2-Jul-2024
https://doi.org/10.1007/978-3-031-64302-6_4
Watanabe KDengel AIshimaru S(2023)Accelerating Knowledge Transfer by Sensing and Actuating Social-Cognitive StatesAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610769(258-262)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1145/3594739.3610769
Ross SMartinez FHoude SMuller MWeisz J(2023)The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software DevelopmentProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584037(491-514)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581641.3584037
Munyaka IAshktorab ZDugan CJohnson JPan Q(2023)Decision Making Strategies and Team Efficacy in Human-AI TeamsProceedings of the ACM on Human-Computer Interaction10.1145/35794767:CSCW1(1-24)Online publication date: 16-Apr-2023
https://dl.acm.org/doi/10.1145/3579476
Seaborn K(2023)Interacting with Masculinities: A Scoping ReviewExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585770(1-12)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585770
Zheng CWu YShi CMa SLuo JMa X(2023)Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-MakingProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581131(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581131
Leoste JPöial JMarmor KFenyvesi KHäkkinen P(2023)Pair-Programming with a Telepresence RobotRobotics in Education10.1007/978-3-031-38454-7_13(143-154)Online publication date: 4-Oct-2023
https://doi.org/10.1007/978-3-031-38454-7_13
Hart JEnsley MKuttal S(2023)Gender Effects on Creativity When Pair Programming with a Human vs. an AgentHCI International 2023 Posters10.1007/978-3-031-35989-7_75(594-602)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35989-7_75
Sardana ASharma V(2023)Agile Framework Adaptation Issues in Various SectorsAgile Software Development10.1002/9781119896838.ch2(23-38)Online publication date: 8-Feb-2023
https://doi.org/10.1002/9781119896838.ch2
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents