Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based Study
Authors: 
Rosalia Tufano, Antonio Mastropaolo, Federica Pepe, Ozren Dabic, Massimiliano Di Penta, and Gabriele BavotaAuthors Info & Claims
April 2024
Pages 571 - 583
Abstract
Large Language Models (LLMs) have gained significant attention in the software engineering community. Nowadays developers have the possibility to exploit these models through industrial-grade tools providing a handy interface toward LLMs, such as OpenAI's ChatGPT. While the potential of LLMs in assisting developers across several tasks has been documented in the literature, there is a lack of empirical evidence mapping the actual usage of LLMs in software projects. In this work, we aim at filling such a gap. First, we mine 1,501 commits, pull requests (PRs), and issues from open-source projects by matching regular expressions likely to indicate the usage of ChatGPT to accomplish the task. Then, we manually analyze these instances, discarding false positives (i.e., instances in which ChatGPT was mentioned but not actually used) and categorizing the task automated in the 467 true positive instances (165 commits, 159 PRs, 143 issues). This resulted in a taxonomy of 45 tasks which developers automate via ChatGPT. The taxonomy, accompanied with representative examples, provides (i) developers with valuable insights on how to exploit LLMs in their workflow and (ii) researchers with a clear overview of tasks that, according to developers, could benefit from automated solutions.
References
[1]
[n. d.]. ChatGPT. https://openai.com/blog/chatgpt. Accessed: 2023-03-27.
[2]
[n. d.]. Copilot Website. https://copilot.github.com. Accessed: 2022-11-10.
[3]
2023. https://github.com/woocommerce/woocommerce/pull/37233.
[4]
2023. https://github.com/fufexan/nix-gaming/pull/73.
[5]
2023. https://github.com/danielgross/whatsapp-gpt/issues/68.
[6]
2023. https://github.com/typescript-eslint/typescript-eslint/pull/6915.
[7]
2023. https://github.com/dogsheep/apple-notes-to-sqlite/issues/1.
[8]
2023. https://github.com/pfusik/cito/issues/80.
[9]
2023. https://github.com/greenshot/greenshot/pull/484.
[10]
2023. https://github.com/vcmi/vcmi/pull/1659.
[11]
2023. https://github.com/garden-io/garden/pull/4553.
[12]
2023. https://github.com/1j01/textual-paint/commit/e9494ddf.
[13]
2023. https://github.com/go-go-golems/glazed/issues/50.
[14]
2023. https://github.com/fluxninja/aperture/commit/70a68635.
[15]
2023. https://github.com/talent-connect/connect/issues/658.
[16]
2023. https://github.com/gofiber/fiber/issues/2301.
[17]
2023. https://github.com/libp2p/js-libp2p/issues/1648.
[18]
2023. https://github.com/spacedriveapp/spacedrive/pull/925.
[19]
2023. https://github.com/pizzaboxer/bloxstrap/issues/224.
[20]
2023. https://github.com/spring-cloud/spring-cloud-stream/issues/2643.
[21]
2023. https://github.com/shaka-project/shaka-player/issues/5015.
[22]
2023. https://github.com/module-federation/module-federation-examples/issues/2942.
[23]
2023. https://github.com/prosyslab-classroom/cs348-information-security/issues/365.
[24]
2023. https://github.com/pbui/bobbit/commit/089fc145.
[25]
2023. https://github.com/dodona-edu/dodona/commit/9efb97f8.
[26]
2023. https://github.com/reorx/jsoncv/commit/1d5f8f1d.
[27]
2023. https://github.com/sirupsen/napkin-math/issues/26.
[28]
2023. https://github.com/hubtype/botonic/pull/2491.
[29]
2023. https://github.com/kvas-it/pytest-console-scripts/pull/76.
[30]
2023. https://github.com/kkdai/chatgpt/pull/4.
[31]
2023. https://github.com/spotlightpa/almanack/commit/955fc76b.
[32]
2023. https://github.com/kohya-ss/sd-webui-additional-networks/issues/43.
[33]
2023. https://github.com/rootzoll/raspiblitz/issues/3640.
[34]
2023. https://github.com/puppeteer/puppeteer/issues/9959.
[35]
2023. https://github.com/kiali/kiali/pull/5973.
[36]
2023. https://github.com/shilomagen/passport-extension/pull/16.
[37]
2023. https://github.com/kyverno/kyverno/pull/5834.
[38]
2023. https://github.com/failfa-st/hyv/pull/1.
[39]
2023. https://github.com/robherley/snips.sh/pull/17.
[40]
2023. https://github.com/pwncollege/dojo/issues/132.
[41]
2023. https://github.com/az-digital/az_quickstart/pull/2226.
[42]
2023. n. https://github.com/igrigorik/videospeed/issues/1035.
[43]
E. Aghajani, G. Bavota, M. Linares-Vásquez, and M. Lanza. 2021. Automated Documentation of Android Apps. IEEE Transactions on Software Engineering, TSE 47, 1 (2021), 204--220.
[44]
Miltiadis Allamanis, Earl T. Barr, Christian Bird, and Charles Sutton. 2015. Suggesting Accurate Method and Class Names. In 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE. 38--49.
[45]
Anonymous. 2023. Replication Package https://github.com/unveilingchatgptsusage/unveilingchatgptsusage.
[46]
Apache Software Foundation. [n. d.]. Guide for new project contributors https://community.apache.org/contributors/. Accessed: 2023-07-08.
[47]
Owura Asare, Meiyappan Nagappan, and N Asokan. 2022. Is github's copilot as bad as humans at introducing vulnerabilities in code? arXiv preprint arXiv:2204.04741 (2022).
[48]
Johannes Bader, Andrew Scott, Michael Pradel, and Satish Chandra. 2019. Getafix: learning to fix bugs automatically. ACM Program. Lang., Object-oriented Programming, Systems, Languages, and Applications OOPSLA (2019), 159:1--159:27.
[49]
Maria Teresa Baldassarre, Danilo Caivano, Davide Fucci, Natalia Juristo, Simone Romano, Giuseppe Scanniello, and Burak Turhan. 2021. Studying test-driven development and its retainment over a six-month time span. J. Syst. Softw. 176 (2021), 110937.
[50]
Carlos Bernal-Cárdenas, Nathan Cooper, Madeleine Havranek, Kevin Moran, Oscar Chaparro, Denys Poshyvanyk, and Andrian Marcus. 2023. Translating Video Recordings of Complex Mobile App UI Gestures into Replayable Scenarios. IEEE Trans. Software Eng. 49, 4 (2023), 1782--1803.
[51]
Jialun Cao, Meiziniu Li, Ming Wen, and Shing-chi Cheung. 2023. A study on prompt design, advantages and limitations of chatgpt for deep learning program repair. arXiv preprint arXiv:2304.08191 (2023).
[52]
Joymallya Chakraborty, Suvodeep Majumder, and Tim Menzies. 2021. Bias in machine learning software: why? how? what to do?. In ESEC/FSE '21: 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 429--440.
[53]
Joymallya Chakraborty, Suvodeep Majumder, Zhe Yu, and Tim Menzies. 2020. Fairway: a way to build fair ML software. In ESEC/FSE '20: 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 654--665.
[54]
Zimin Chen, Steve Kommrusch, and Martin Monperrus. 2023. Neural Transfer Learning for Repairing Security Vulnerabilities in C Code. IEEE Transactions on Software Engineering 49, 1 (2023), 147--165.
[55]
Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso. 2011. WATER: Web Application TEst Repair. In Proceedings of the First International Workshop on End-to-End Test Script Engineering (ETSE '11). Association for Computing Machinery, 24--29.
[56]
Matteo Ciniselli, Nathan Cooper, Luca Pascarella, Antonio Mastropaolo, Emad Aghajani, Denys Poshyvanyk, Massimiliano Di Penta, and Gabriele Bavota. 2021. An Empirical Study on the Usage of Transformer Models for Code Completion. IEEE Transactions on Software Engineering, TSE 48, 12 (2021), 4818--4837.
[57]
Adelina Ciurumelea, Andreas Schaufelbühl, Sebastiano Panichella, and Harald C. Gall. 2017. Analyzing reviews and code of mobile apps for better release planning. In IEEE 24th International Conference on Software Analysis, Evolution and Reengineering, SANER. IEEE Computer Society, 91--102.
[58]
J Cohen. 1960. A coefficient of agreement for nominal scales. Educ Psychol Meas. (1960).
[59]
Brett Daniel, Tihomir Gvero, and Darko Marinov. 2010. On Test Repair Using Symbolic Execution. In Proceedings of the 19th International Symposium on Software Testing and Analysis (ISSTA '10). Association for Computing Machinery, 207--218.
[60]
Jinhao Dong, Yiling Lou, Qihao Zhu, Zeyu Sun, Zhilin Li, Wenjie Zhang, and Dan Hao. 2022. FIRA: Fine-Grained Graph-Based Code Change Representation for Automated Commit Message Generation. In Proceedings of the 44th International Conference on Software Engineering (ICSE '22). Association for Computing Machinery, 970--981.
[61]
Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. 2023. Self-collaboration Code Generation via ChatGPT. arXiv preprint arXiv:2304.07590 (2023).
[62]
Eclipse Foundation. [n. d.]. Platform/How to contribute https://wiki.eclipse.org/Platform/How_to_Contribute. Accessed: 2023-07-08.
[63]
Camilo Escobar-Velásquez, Michael Osorio-Riaño, Juan Dominguez-Osorio, Maria Arevalo, and Mario Linares-Vásquez. 2020. An Empirical Study of i18n Collateral Changes and Bugs in GUIs of Android apps. In 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). 581--592.
[64]
Zhiyu Fan, Xiang Gao, Abhik Roychoudhury, and Shin Hwei Tan. 2022. Automated Repair of Programs from Large Language Models. arXiv preprint arXiv:2205.10583 (2022).
[65]
Sen Fang, Tao Zhang, You-Shuai Tan, Zhou Xu, Zhi-Xin Yuan, and Ling-Ze Meng. 2022. PRHAN: Automated Pull Request Description Generation Based on Hybrid Attention Network. Journal of Systems and Software 185 (2022), 111160.
[66]
Mattia Fazzini, Kevin Moran, Carlos Bernal-Cárdenas, Tyler Wendland, Alessandro Orso, and Denys Poshyvanyk. 2023. Enhancing Mobile App Bug Reporting via Real-Time Understanding of Reproduction Steps. IEEE Trans. Software Eng. 49, 3 (2023), 1246--1272.
[67]
Davide Fucci, Hakan Erdogmus, Burak Turhan, Markku Oivo, and Natalia Juristo. 2017. A Dissection of the Test-Driven Development Process: Does It Really Matter to Test-First or to Test-Last? IEEE Trans. Software Eng. 43, 7 (2017), 597--614.
[68]
Xiang Gao, Bo Wang, Gregory J. Duck, Ruyi Ji, Yingfei Xiong, and Abhik Roychoudhury. 2021. Beyond Tests: Program Vulnerability Repair via Crash Constraint Extraction. ACM Trans. Softw. Eng. Methodol. 30, 2 (2021).
[69]
Inc. Google. 2023. Try Bard, an AI expertiment by Google https://bard.google.com.
[70]
Z. Han, X. Li, Z. Xing, H. Liu, and Z. Feng. 2017. Learning to Predict Severity of Software Vulnerability Using Only Vulnerability Description. In 33th IEEE International Conference on Software Maintenance and Evolution ICSME. 125--136.
[71]
Saki Imai. 2022. Is GitHub Copilot a Substitute for Human Pair-programming? An Empirical Study. In 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 319--321.
[72]
Ivana Clairine Irsan, Ting Zhang, Ferdian Thung, David Lo, and Lingxiao Jiang. 2022. AutoPRTitle: A Tool for Automatic Pull Request Title Generation. In 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME). 454--458.
[73]
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hallucination in Natural Language Generation. ACM Comput. Surv. (2023).
[74]
Siyuan Jiang, Ameer Armaly, and Collin McMillan. 2017. Automatically generating commit messages from diffs using neural machine translation. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). 135--146.
[75]
Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems.
[76]
Zhiyu Li, Shuai Lu, Daya Guo, Nan Duan, Shailesh Jannu, Grant Jenks, Deep Majumder, Jared Green, Alexey Svyatkovskiy, Shengyu Fu, and Neel Sundaresan. 2022. Automating code review activities by large-scale pre-training. In 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE. 1035--1047.
[77]
Zhongxin Liu, Xin Xia, Ahmed E. Hassan, David Lo, Zhenchang Xing, and Xinyu Wang. 2018. Neural-Machine-Translation-Based Commit Message Generation: How Far Are We?. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE '18). Association for Computing Machinery, 373--384.
[78]
Zhongxin Liu, Xin Xia, Ahmed E. Hassan, David Lo, Zhenchang Xing, and Xinyu Wang. 2018. Neural-machine-translation-based commit message generation: how far are we?. In 33rd IEEE/ACM International Conference on Automated Software Engineering, ASE. 373--384.
[79]
Zhongxin Liu, Xin Xia, Christoph Treude, David Lo, and Shanping Li. 2019. Automatic Generation of Pull Request Descriptions. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). 176--188.
[80]
Aniketh Malyala, Katelyn Zhou, Baishakhi Ray, and Saikat Chakraborty. 2023. On ML-Based Program Translation: Perils and Promises. In 45th International Conference on Software Engineering, ICSE '23, Companion Proceedings, 2023.
[81]
Ehsan Mashhadi and Hadi Hemmati. 2021. Applying CodeBERT for Automated Program Repair of Java Simple Bugs. In 18th IEEE/ACM International Conference on Mining Software Repositories, MSR. 505--509.
[82]
Antonio Mastropaolo, Luca Pascarella, Emanuela Guglielmi, Matteo Ciniselli, Simone Scalabrino, Rocco Oliveto, and Gabriele Bavota. 2023. On the robustness of code generation techniques: An empirical study on github copilot. arXiv preprint arXiv:2302.00438 (2023).
[83]
Laura Moreno, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Andrian Marcus. 2015. How Can I Use This Method?. In 37th IEEE/ACM International Conference on Software Engineering, ICSE. 880--890.
[84]
Nathalia Nascimento, Paulo Alencar, and Donald Cowan. 2023. Comparing Software Developers with ChatGPT: An Empirical Investigation. arXiv preprint arXiv:2305.11837 (2023).
[85]
Anh Tuan Nguyen, Hoan Anh Nguyen, Tung Thanh Nguyen, and Tien N. Nguyen. 2014. Statistical Learning Approach for Mining API Usage Mappings for Code Migration. In 29th IEEE/ACM International Conference on Automated Software Engineering, ASE. 457--468.
[86]
Anh Tuan Nguyen, Tung Thanh Nguyen, Hoan Anh Nguyen, Ahmed Tamrawi, Hung Viet Nguyen, Jafar M. Al-Kofahi, and Tien N. Nguyen. 2012. Graph-based pattern-oriented, context-sensitive source code completion. In 34th IEEE/ACM International Conference on Software Engineering, ICSE. 69--79.
[87]
Anh Tuan Nguyen, Tung Thanh Nguyen, and Tien N. Nguyen. 2014. Migrating Code with Statistical Machine Translation. In 36th IEEE/ACM International Conference on Software Engineering, ICSE. 544--547.
[88]
Nhan Nguyen and Sarah Nadi. 2022. An Empirical Evaluation of GitHub Copilot's Code Suggestions. In 2022 IEEE/ACM 19th International Conference on Mining Software Repositories (MSR). IEEE, 1--5.
[89]
Phuong T. Nguyen, Claudio Di Sipio, Juri Di Rocco, Massimiliano Di Penta, and Davide Di Ruscio. 2021. Adversarial Attacks to API Recommender Systems: Time to Wake Up and Smell the Coffee f. In 36th IEEE/ACM International Conference on Automated Software Engineering, ASE 2021. 253--265.
[90]
Minxue Pan, Tongtong Xu, Yu Pei, Zhong Li, Tian Zhang, and Xuandong Li. 2022. GUI-Guided Test Script Repair for Mobile Apps. IEEE Transactions on Software Engineering 48, 3 (2022), 910--929.
[91]
Sebastiano Panichella, Andrea Di Sorbo, Emitza Guzman, Corrado Aaron Visaggio, Gerardo Canfora, and Harald C. Gall. 2015. How can i improve my app? Classifying user reviews for software maintenance and evolution. In IEEE International Conference on Software Maintenance and Evolution, ICSME. IEEE Computer Society, 281--290.
[92]
Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2021. An Empirical Cybersecurity Evaluation of GitHub Copilot's Code Contributions. arXiv preprint arXiv:2108.09293 (2021).
[93]
Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590 (2023).
[94]
Martin P. Robillard, Robert J. Walker, and Thomas Zimmermann. 2010. Recommendation Systems for Software Engineering. IEEE Softw. 27, 4 (2010), 80--86.
[95]
Simone Scalabrino, Gabriele Bavota, Barbara Russo, Massimiliano Di Penta, and Rocco Oliveto. 2019. Listening to the Crowd for the Release Planning of Mobile Apps. IEEE Trans. Software Eng. 45, 1 (2019), 68--86.
[96]
Dominik Sobania, Martin Briesch, Carol Hanna, and Justyna Petke. 2023. An analysis of the automatic bug fixing performance of chatgpt. arXiv preprint arXiv:2301.08653 (2023).
[97]
Dominik Sobania, Martin Briesch, and Franz Rothlauf. 2021. Choose Your Programming Copilot: A Comparison of the Program Synthesis Performance of GitHub Copilot and Genetic Programming. arXiv preprint arXiv:2111.07875 (2021).
[98]
Donna Spencer. 2009. Card sorting: Designing usable categories. Rosenfeld Media.
[99]
Giriprasad Sridhara, Sourav Mazumdar, et al. 2023. ChatGPT: A Study on its Utility for Ubiquitous Software Engineering Tasks. arXiv preprint arXiv:2305.16837 (2023).
[100]
Giriprasad Sridhara, Lori Pollock, and K Vijay-Shanker. 2011. Automatically detecting and describing high level actions within methods. In 33rd IEEE/ACM International Conference on Software Engineering, ICSE. 101--110.
[101]
Igor Steinmacher, Tayana Uchôa Conte, Christoph Treude, and Marco Aurélio Gerosa. 2016. Overcoming open source project entry barriers with a portal for newcomers. In Proceedings of the 38th International Conference on Software Engineering, ICSE 2016. ACM, 273--284.
[102]
Andrea Stocco, Rahulkrishna Yandrapally, and Ali Mesbah. 2018. Visual Web Test Repair. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018). Association for Computing Machinery, 503--514.
[103]
Alexey Svyatkovskiy, Shao Kun Deng, Shengyu Fu, and Neel Sundaresan. 2020. IntelliCode compose: code generation using transformer. In 28th ACM Joint European Software Engineering Conference and the ACM/SIGSOFT International Symposium on the Foundations of Software Engineering ESEC-FSE. 1433--1443.
[104]
Patanamon Thongtanunam, Chanathip Pornprasit, and Chakkrit Tantithamthavorn. 2022. AutoTransform: Automated Code Transformation to Support Modern Code Review Process. In 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE). 237--248.
[105]
Haoye Tian, Weiqi Lu, Tsz On Li, Xunzhu Tang, Shing-Chi Cheung, Jacques Klein, and Tegawendé F Bissyandé. 2023. Is ChatGPT the Ultimate Programming Assistant-How far is it? arXiv preprint arXiv:2304.11938 (2023).
[106]
Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. 2019. An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation. ACM Trans. Softw. Eng. Methodol. 28, 4 (2019), 19:1--19:29.
[107]
Rosalia Tufano, Simone Masiero, Antonio Mastropaolo, Luca Pascarella, Denys Poshyvanyk, and Gabriele Bavota. 2022. Using Pre-Trained Models to Boost Code Review Automation. In 44th IEEE/ACM International Conference on Software Engineering, ICSE. 2291--2302.
[108]
Rosalia Tufano, Luca Pascarella, Michele Tufano, Denys Poshyvanyk, and Gabriele Bavota. 2021. Towards Automating Code Review Activities. In 43rd IEEE/ACM International Conference on Software Engineering, ICSE. 163--174.
[109]
Priyan Vaithilingam, Tianyi Zhang, and Elena L Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1--7.
[110]
Haoye Wang, Xin Xia, David Lo, Qiang He, Xinyu Wang, and John Grundy. 2021. Context-Aware Retrieval-Based Deep Commit Message Generation. ACM Trans. Softw. Eng. Methodol. (2021), 30 pages.
[111]
Fengcai Wen, Emad Aghajani, Csaba Nagy, Michele Lanza, and Gabriele Bavota. 2021. Siri, Write the Next Method. In 43rd IEEE/ACM International Conference on Software Engineering, ICSE. 138--149.
[112]
Jules White, Sam Hays, Quchen Fu, Jesse Spencer-Smith, and Douglas C Schmidt. 2023. Chatgpt prompt patterns for improving code quality, refactoring, requirements elicitation, and software design. arXiv preprint arXiv:2303.07839 (2023).
[113]
Dakota Wong, Austin Kothig, and Patrick Lam. 2022. Exploring the Verifiability of Code Generated by GitHub Copilot. arXiv preprint arXiv:2209.01766 (2022).
[114]
Xin Xia, David Lo, Ying Ding, Jafar M. Al-Kofahi, Tien N. Nguyen, and Xinyu Wang. 2017. Improving Automated Bug Triaging with Specialized Topic Model. IEEE Trans. Software Eng. 43, 3 (2017), 272--297.
[115]
Burak Yetistiren, Isik Ozsoy, and Eray Tuzun. 2022. Assessing the quality of GitHub copilot's code generation. In Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering. 62--71.
[116]
Yu Zhao, Ting Su, Yang Liu, Wei Zheng, Xiaoxue Wu, Ramakanth Kavuluru, William G. J. Halfond, and Tingting Yu. 2022. ReCDroid+: Automated End-to-End Crash Reproduction from Bug Reports for Android Apps. ACM Trans. Softw. Eng. Methodol. 31, 3 (2022), 36:1--36:33.
[117]
Minghui Zhou and Audris Mockus. 2010. Growth of newcomer competence: challenges of globalization. In Proceedings of the Workshop on Future of Software Engineering Research, FoSER 2010, at the 18th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 443--448.
[118]
Albert Ziegler, Eirini Kalliamvakou, X. Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittampalam, and Edward Aftandilian. 2022. Productivity assessment of neural code completion. In International Symposium on Machine Programming. 21--29.
Recommendations
Chatting with AI: Deciphering Developer Conversations with ChatGPT
MSR '24: Proceedings of the 21st International Conference on Mining Software RepositoriesLarge Language Models (LLMs) have been widely adopted and are becoming ubiquitous and integral to software development. However, we have little knowledge as to how these tools are being used by software developers beyond anecdotal evidence and word-of-...
A Preliminary Study of Bots Usage in Open Source Community
Internetware '22: Proceedings of the 13th Asia-Pacific Symposium on InternetwareBots are seen as a promising approach in software development, which help to deal with the ever-increasing complexity of modern software engineering and development. The number of bots in open source community, such as GitHub, has expanded ...
Test automation in open-source Android apps: a large-scale empirical study
ASE '20: Proceedings of the 35th IEEE/ACM International Conference on Automated Software EngineeringAutomated testing of mobile apps has received significant attention in recent years from researchers and practitioners alike. In this paper, we report on the largest empirical study to date, aimed at understanding the test automation culture prevalent ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2024
788 pages
ISBN:9798400705878
DOI:10.1145/3643991
- Chair:
- Diomidis Spinellis,
- Program Chair:
- Alberto Bacchelli,
- Program Co-chair:
- Eleni Constantinou
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- European Research Council
- PRIN 2020
Conference
MSR '24
Sponsor:
MSR '24: 21st International Conference on Mining Software Repositories
April 15 - 16, 2024
Lisbon, Portugal
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 16Total Downloads
- Downloads (Last 12 months)16
- Downloads (Last 6 weeks)16
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in