What Do Developers Feel About Fast-Growing Programming Languages? An Exploratory Study
Pages 178 - 189
Abstract
The developer community has witnessed an unprecedented surge in recent years, with over 100 million active developers on the GitHub platform in 2023. Along with it, there is a significant rise and adoption of new programming languages, frameworks and tools. The study aims to comprehend how developers perceive these fast-growing programming languages by performing emotion analysis of developer's comments posted in various software artifacts such as pull requests, issues and commits of GitHub repositories. In this regard, we employed a fine-tuned small 'Large Language Model' (sLLM) to detect emotions, leveraging a balanced dataset from existing literature complemented with additional manual annotations from our collected data. We have analyzed 10 fast-growing programming languages, examining 1.8 million comments from 4.1 million non-code artifacts. To further validate our findings, we have performed a qualitative survey and analysis with 28 developers. Our study reveals insights into the developers emotion associated with these fast-growing languages. Notably, "Surprise" is the predominant emotion associated with these languages.
References
[1]
Marvin M Agüero-Torales, José I Abreu Salas, and Antonio G López-Herrera. 2021. Deep learning and multilingual sentiment analysis on social media data: An overview. Applied Soft Computing 107 (2021), 107373.
[2]
Toufique Ahmed and Premkumar Devanbu. 2022. Few-shot training LLMs for project-specific code-summarization. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. 1--5.
[3]
Toluwalase Awoyemi, Kayode E Ogunniyi, Adedolapo V Adejumo, Ujunwa Ebili, Abiola Olusanya, Eloho H Olojakpoke, Olufunto Shonibare, and Olufunto O Shonibare. 2022. Emotional Analysis of Tweets About Clinically Extremely Vulnerable COVID-19 Groups. Cureus 14, 9 (2022).
[4]
Sarmad Bashir, Muhammad Abbas, Mehrdad Saadatmand, Eduard Paul Enoiu, Markus Bohlin, and Pernilla Lindberg. 2023. Requirement or not, that is the question: A case from the railway industry. In International Working Conference on Requirements Engineering: Foundation for Software Quality. Springer, 105--121.
[5]
Gilbert Louis Bernstein and Fredrik Kjolstad. 2016. Perspectives: why new programming languages for simulation? ACM Transactions on Graphics (TOG) 35, 2 (2016), 1--3.
[6]
Donald Bleyl and Elham Khorasani Buxton. 2022. Emotion Recognition on StackOverflow Posts Using BERT. In 2022 IEEE International Conference on Big Data (Big Data). IEEE, 5881--5885.
[7]
Fabio Calefato, Filippo Lanubile, and Nicole Novielli. 2017. Emotxt: a toolkit for emotion recognition from text. In 2017 seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). IEEE, 79--80.
[8]
Gul Calikli, Mohammed Al-Eryani, Emil Baldebo, Jennifer Horkofff, and Alexander Ask. 2018. Effects of automated competency evaluation on software engineers' emotions and motivation: a case study. In Proceedings of the 3rd International Workshop on Emotion Awareness in Software Engineering. 44--50.
[9]
Meriem Ben Chaaben, Lola Burgueño, and Houari Sahraoui. 2023. Towards using few-shot prompt learning for automating model completion. In 2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER). IEEE, 7--12.
[10]
Robert Chatley, Alastair Donaldson, and Alan Mycroft. 2019. The next 7000 programming languages. Computing and software science: State of the art and perspectives (2019), 250--282.
[11]
Debarshi Chatterji, Jeffrey C Carver, Nicholas A Kraft, and Jan Harder. 2013. Effects of cloned code on software maintainability: A replicated developer study. In 2013 20th Working Conference on Reverse Engineering (WCRE). IEEE, 112--121.
[12]
Mihai Codoban, Sruti Srinivasa Ragavan, Danny Dig, and Brian Bailey. 2015. Software history under the lens: A study on why and how developers examine it. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 1--10.
[13]
Jailton Coelho, Marco Tulio Valente, Luciano Milen, and Luciana L Silva. 2020. Is this GitHub project maintained? Measuring the level of maintenance activity of open-source projects. Information and Software Technology 122 (2020), 106274.
[14]
Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37--46.
[15]
Stefano Comino, Fabio M Manenti, and Maria Laura Parisi. 2007. From planning to mature: On the success of open source projects. Research policy 36, 10 (2007), 1575--1586.
[16]
Roland Croft, Yongzheng Xie, Mansooreh Zahedi, M Ali Babar, and Christoph Treude. 2022. An empirical study of developers' discussions about security challenges of different programming languages. Empirical Software Engineering 27 (2022), 1--52.
[17]
Ajoy Das, Gias Uddin, and Guenther Ruhe. 2022. An empirical study of blockchain repositories in GitHub. In Proceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering. 211--220.
[18]
Paul Ekman. 1992. An argument for basic emotions. Cognition & emotion 6, 3--4 (1992), 169--200.
[19]
Emily Escamilla, Martin Klein, Talya Cooper, Vicky Rampin, Michele C Weigle, and Michael L Nelson. 2022. The Rise of GitHub in Scholarly Publications. In International Conference on Theory and Practice of Digital Libraries. Springer, 187--200.
[20]
Denae Ford, Margaret-Anne Storey, Thomas Zimmermann, Christian Bird, Sonia Jaffe, Chandra Maddila, Jenna L Butler, Brian Houck, and Nachiappan Nagappan. 2021. A tale of two cities: Software developers working from home during the covid-19 pandemic. ACM Transactions on Software Engineering and Methodology (TOSEM) 31, 2 (2021), 1--37.
[21]
Amir Ghorbani, Nathan Cassee, Derek Robinson, Adam Alami, Neil A Ernst, Alexander Serebrenik, and Andrzej Wąsowski. 2023. Autonomy Is An Acquired Taste: Exploring Developer Preferences for GitHub Bots. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1405--1417.
[22]
Daniela Girardi, Nicole Novielli, Davide Fucci, and Filippo Lanubile. 2020. Recognizing developers' emotions while programming. In Proceedings of the ACM/IEEE 42nd International Conference on software Engineering. 666--677.
[23]
Frederick J Gravetter and Lori-Ann B Forzano. 2018. Research methods for the behavioral sciences. Cengage learning.
[24]
Daniel Graziotin, Fabian Fagerholm, Xiaofeng Wang, and Pekka Abrahamsson. 2017. On the unhappiness of software developers. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering. 324--333.
[25]
Emitza Guzman, David Azócar, and Yang Li. 2014. Sentiment analysis of commit comments in GitHub: an empirical study. In Proceedings of the 11th working Conference on Mining Software Repositories. 352--355.
[26]
Roman Haas, Daniel Elsner, Elmar Juergens, Alexander Pretschner, and Sven Apel. 2021. How can manual testing processes be optimized? developer survey, optimization guidelines, and case studies. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1281--1291.
[27]
Mohammad A Hadi and Fatemeh H Fard. 2023. Evaluating pre-trained models for user feedback analysis in software engineering: A study on classification of app-reviews. Empirical Software Engineering 28, 4 (2023), 88.
[28]
Jochen Hartmann. 2022. Emotion English DistilRoBERTa-base. https://huggingface.co/j-hartmann/emotion-english-distilroberta-base/.
[29]
Mia Mohammad Imran, Yashasvi Jain, Preetha Chatterjee, and Kostadin Damevski. 2022. Data Augmentation for Improving Emotion Recognition in Software Engineering Communication. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. 1--13.
[30]
Robbert Jongeling, Subhajit Datta, and Alexander Serebrenik. 2015. Choosing your weapons: On sentiment analysis tools for software engineering research. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 531--535.
[31]
Rohan Kamath, Arpan Ghoshal, Sivaraman Eswaran, and Prasad Honnavalli. 2022. An enhanced context-based emotion detection model using roberta. In 2022 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT). IEEE, 1--6.
[32]
Chris Kim, Uta Hinrichs, Saif M Mohammad, and Christopher Collins. 2020. Lexichrome: Text construction and lexical discovery with word-color associations using interactive visualization. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. 477--488.
[33]
Barbara A Kitchenham, Shari Lawrence Pfleeger, Lesley M Pickard, Peter W Jones, David C. Hoaglin, Khaled El Emam, and Jarrett Rosenberg. 2002. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering 28, 8 (2002), 721--734.
[34]
Karina Kohl, Bogdan Vasilescu, and Rafael Prikladnicki. 2020. Multitasking across industry projects: a replication study. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops. 93--100.
[35]
Aman Kumar, Manish Khare, and Saurabh Tiwari. 2022. Sentiment Analysis of Developers' Comments on GitHub Repository: A Study. In 2022 14th International Conference on Advanced Computational Intelligence (ICACI). IEEE, 91--98.
[36]
J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159--174.
[37]
Luigi Lavazza, Sandro Morasca, and Davide Tosi. 2016. An empirical study on the effect of programming languages on productivity. In Proceedings of the 31st Annual ACM Symposium on Applied Computing. 1434--1439.
[38]
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019).
[39]
Tom Mens. 2016. An ecosystemic and socio-technical view on Software Maintenance and Evolution. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 1--8.
[40]
Alessandro Murgia, Marco Ortu, Parastou Tourani, Bram Adams, and Serge Demeyer. 2018. An exploratory qualitative and quantitative analysis of emotions in issue report comments of open source systems. Empirical Software Engineering 23 (2018), 521--564.
[41]
Alessandro Murgia, Parastou Tourani, Bram Adams, and Marco Ortu. 2014. Do developers feel emotions? an exploratory analysis of emotions in software artifacts. In Proceedings of the 11th working Conference on Mining Software Repositories. 262--271.
[42]
Noor Nashid, Mifta Sintaha, and Ali Mesbah. 2023. Retrieval-based prompt selection for code-related few-shot learning. In Proceedings of the 45th International Conference on Software Engineering (ICSE'23).
[43]
László Nemes and Attila Kiss. 2021. Social media sentiment analysis based on COVID-19. Journal of Information and Telecommunication 5, 1 (2021), 1--15.
[44]
Nicole Novielli, Fabio Calefato, and Filippo Lanubile. 2018. A gold standard for emotion annotation in stack overflow. In Proceedings of the 15th International Conference on Mining Software Repositories. 14--17.
[45]
Marco Ortu, Bram Adams, Giuseppe Destefanis, Parastou Tourani, Michele Marchesi, and Roberto Tonelli. 2015. Are bullies more productive? Empirical study of affectiveness vs. issue fixing time. In 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. IEEE, 303--313.
[46]
Marco Ortu, Giuseppe Destefanis, Steve Counsell, Stephen Swift, Roberto Tonelli, and Michele Marchesi. 2017. How diverse is your team? Investigating gender and nationality diversity in GitHub teams. Journal of Software Engineering Research and Development 5, 1 (2017), 1--18.
[47]
Marco Ortu, Tracy Hall, Michele Marchesi, Roberto Tonelli, David Bowes, and Giuseppe Destefanis. 2018. Mining communication patterns in software development: A github analysis. In Proceedings of the 14th International Conference on predictive models and data analytics in Software Engineering. 70--79.
[48]
Marco Ortu, Alessandro Murgia, Giuseppe Destefanis, Parastou Tourani, Roberto Tonelli, Michele Marchesi, and Bram Adams. 2016. The emotional side of software developers in JIRA. In Proceedings of the 13th International Conference on Mining Software Repositories. 480--483.
[49]
Sunghong Park, Junhee Cho, Kanghee Park, and Hyunjung Shin. 2021. Customer sentiment analysis with more sensibility. Engineering Applications of Artificial Intelligence 104 (2021), 104356.
[50]
Celeste Lyn Paul. 2008. A modified delphi approach to a new card sorting methodology. Journal of Usability studies 4, 1 (2008), 7--30.
[51]
Linda Dailey Paulson. 2007. Developers shift to dynamic programming languages. Computer 40, 2 (2007), 12--15.
[52]
Norman Peitek, Janet Siegmund, Chris Parnin, Sven Apel, Johannes C Hofmeister, and André Brechmann. 2018. Simultaneous measurement of program comprehension with fmri and eye tracking: A case study. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. 1--10.
[53]
Daniel Pletea, Bogdan Vasilescu, and Alexander Serebrenik. 2014. Security and emotion: sentiment analysis of security discussions on github. In Proceedings of the 11th working Conference on Mining Software Repositories. 348--351.
[54]
Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu. 2014. A large scale study of programming languages and code quality in github. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 155--165.
[55]
Amirali Sajadi, Kostadin Damevski, and Preetha Chatterjee. 2023. Interpersonal Trust in OSS: Exploring Dimensions of Trust in GitHub Pull Requests. In 2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER). IEEE, 19--24.
[56]
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019).
[57]
Philip Sedgwick. 2013. Convenience sampling. Bmj 347 (2013).
[58]
Phillip Shaver, Judith Schwartz, Donald Kirson, and Cary O'connor. 1987. Emotion knowledge: further exploration of a prototype approach. Journal of personality and social psychology 52, 6 (1987), 1061.
[59]
Nischal Shrestha, Colton Botta, Titus Barik, and Chris Parnin. 2020. Here we go again: Why is it difficult for developers to learn another programming language?. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 691--701.
[60]
Navdeep Singh and Paramvir Singh. 2017. How do code refactoring activities impact software developers' sentiments?-an empirical investigation into github commits. In 2017 24th Asia-Pacific Software Engineering Conference (APSEC). IEEE, 648--653.
[61]
Donna Spencer and Todd Warfel. 2004. Card sorting: a definitive guide. Boxes and arrows 2, 2004 (2004), 1--23.
[62]
Margaret-Anne Storey, Leif Singer, Brendan Cleary, Fernando Figueira Filho, and Alexey Zagalsky. 2014. The (r) evolution of social media in software engineering. Future of Software Engineering proceedings (2014), 100--116.
[63]
Luisa Stracqualursi and Patrizia Agati. 2022. Tweet topics and sentiments relating to distance learning among Italian Twitter users. Scientific reports 12, 1 (2022), 9163.
[64]
Jirateep Tantisuwankul, Yusuf Sulistyo Nugroho, Raula Gaikovina Kula, Hideaki Hata, Arnon Rungsawang, Pattara Leelaprute, and Kenichi Matsumoto. 2019. A topological analysis of communication channels for knowledge sharing in contemporary GitHub projects. Journal of Systems and Software 158 (2019), 110416.
[65]
Tien Rahayu Tulili, Andrea Capiluppi, and Ayushi Rastogi. 2022. Burnout in software engineering: A systematic mapping study. Information and Software Technology (2022), 107116.
[66]
Gias Uddin, Yann-Gaël Guéhénuc, Foutse Khomh, and Chanchal K Roy. 2022. An empirical study of the effectiveness of an ensemble of stand-alone sentiment detection tools for software engineering datasets. ACM Transactions on Software Engineering and Methodology (TOSEM) 31, 3 (2022), 1--38.
[67]
Akhila Sri Manasa Venigalla and Sridhar Chimalakonda. 2021. Understanding emotions of developer community towards software documentation. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS). IEEE, 87--91.
[68]
Robert F Woolson. 2007. Wilcoxon signed-rank test. Wiley encyclopedia of clinical trials (2007), 1--3.
[69]
Michal R Wrobel. 2013. Emotions in the software development process. In 2013 6th International Conference on Human System Interactions (HSI). IEEE, 518--523.
[70]
Bo Yang, Xinjie Wei, and Chao Liu. 2017. Sentiments analysis in GitHub repositories: An empirical study. In 2017 24th Asia-Pacific Software Engineering Conference Workshops (APSECW). IEEE, 84--89.
[71]
Jerrold H Zar. 1972. Significance testing of the Spearman rank correlation coefficient. J. Amer. Statist. Assoc. 67, 339 (1972), 578--580.
Index Terms
- What Do Developers Feel About Fast-Growing Programming Languages? An Exploratory Study
Recommendations
Programming Languages The First 25 Years
The programming language field is certainly one of the most important subfields of computer science. It is rich in concepts, theories, and practical developments. The present paper attempts to trace the 25 year development of programming languages by ...
Do developers feel emotions? an exploratory analysis of emotions in software artifacts
MSR 2014: Proceedings of the 11th Working Conference on Mining Software RepositoriesSoftware development is a collaborative activity in which developers interact to create and maintain a complex software system. Human collaboration inevitably evokes emotions like joy or sadness, which can affect the collaboration either positively or ...
A simple separate compilation mechanism for block-structured languages
A very simple and efficient technique for the introduction of separate compilation facilities into compilers for block-structured languages is presented. Using this technique, programs may be compiled in parts while the compile-time checking advantages ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2024
487 pages
ISBN:9798400705861
DOI:10.1145/3643916
- Chair:
- Igor Steinmacher,
- Co-chair:
- Mario Linares-Vasquez,
- Program Chair:
- Kevin Patrick Moran,
- Program Co-chair:
- Olga Baysal
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
ICPC '24
Sponsor:
ICPC '24: 32nd IEEE/ACM International Conference on Program Comprehension
April 15 - 16, 2024
Lisbon, Portugal
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 63Total Downloads
- Downloads (Last 12 months)63
- Downloads (Last 6 weeks)25
Reflects downloads up to 14 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in