Is github copilot a substitute for human pair-programming? an empirical study

S Imai - Proceedings of the ACM/IEEE 44th International …, 2022 - dl.acm.org
Proceedings of the ACM/IEEE 44th International Conference on Software …, 2022dl.acm.org
This empirical study investigates the effectiveness of pair programming with GitHub Copilot
in comparison to human pair-programming. Through an experiment with 21 participants we
focus on code productivity and code quality. For experimental design, a participant was
given a project to code, under three conditions presented in a randomized order. The
conditions are pair-programming with Copilot, human pair-programming as a driver, and as
a navigator. The codes generated from the three trials were analyzed to determine how …
This empirical study investigates the effectiveness of pair programming with GitHub Copilot in comparison to human pair-programming. Through an experiment with 21 participants we focus on code productivity and code quality. For experimental design, a participant was given a project to code, under three conditions presented in a randomized order. The conditions are pair-programming with Copilot, human pair-programming as a driver, and as a navigator. The codes generated from the three trials were analyzed to determine how many lines of code on average were added in each condition and how many lines of code on average were removed in the subsequent stage. The former measures the productivity of each condition while the latter measures the quality of the produced code. The results suggest that although Copilot increases productivity as measured by lines of code added, the quality of code produced is inferior by having more lines of code deleted in the subsequent trial.
ACM Digital Library