research-article

PRHAN: : Automated Pull Request Description Generation Based on Hybrid Attention Network

Authors:

Ling-Ze MengAuthors Info & Claims

Volume 185, Issue C

https://doi.org/10.1016/j.jss.2021.111160

Published: 01 March 2022 Publication History

Abstract

Descriptions of pull requests (PRs) are posted by developers for describing the modifications that they have made and the corresponding reasons in these PRs. Although PRs help developers improve the development efficiency, some developers usually ignore writing the descriptions for PRs. To alleviate the above problem, researchers generally utilize text summarization model to automatically generate descriptions for PRs. However, current RNN-based models still face the challenges such as low efficiency and out-of-vocabulary (OOV), which may influence the further performance improvement to their models. To break this bottleneck, we propose a novel model aiming at the above challenges, named PRHAN (Pull Requests Description Generation Based on Hybrid Attention Network). Specifically, the core of PRHAN is the hybrid attention network, which has faster execution efficiency than RNN-based model. Moreover, we address the OOV problem by the utilizing byte-pair encoding algorithm to build a vocabulary at the sub-word level. Such a vocabulary can represent the OOV words by combining sub-word units. To reduce the sensitivity of the model, we take a simple but effective method into the cross-entropy loss function, named label smoothing. We choose three baseline models, including LeadCM, Transformer and the state-of-the-art model built by Liu et al. and evaluate all the models on the open-source dataset through ROUGE, BLEU, and human evaluation. The experimental results demonstrate that PRHAN is more effective than baselines. Moreover, PRHAN can execute faster than the state-of-the-art model proposed by Liu et al.

Highlights

•

Proposing PRHAN, a hybrid attention based model for PRs descriptions generation.

•

Using BPE to build an open vocabulary to address the out-of-vocabulary problem.

•

Using label smoothing to enforce the generalization of PRHAN.

•

Evaluating the effectiveness of PRHAN by conducting extensive experiments.

References

[1]

Abebe S.L., Ali N., Hassan A.E., An empirical study of software release notes, Empir. Softw. Eng. (EMSE) 21 (2016) 1107–1142.

[2]

Ahmad W.U., Chakraborty S., Ray B., Chang K.-W., A transformer-based approach for source code summarization, 2020, arXiv:2005.00653.

[3]

Bahdanau D., Cho K., Bengio Y., Neural machine translation by jointly learning to align and translate, 2016, arXiv:1409.0473.

[4]

Bauer, L., Wang, Y., Bansal, M., 2018. commonsense for generative multi-hop question answering tasks. In: Proceedings of the 2018 Conference on Empirical Methods in Natural, EMNLP, pp. 4220–4230.

[5]

Conneau, A., Schwenk, H., Barrault, L., LeCun, Y., 2017. very deep convolutional networks for text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL, pp. 1107–1116.

[6]

Cortés-Coy, L.F., Linares-Vásquez, M., Aponte, J., Poshyvanyk, D., 2014. On automatically generating commit messages via summarization of source code changes. In: Proceedings of the 14th IEEE International Working Conference on Source Code Analysis and Manipulation, pp. 275–284.

[7]

Devlin, J., Chang, M., Lee, K., Toutanova, K., 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies NAACL-HLT, pp. 4171–4186.

[8]

Di Sorbo, A., Panichella, S., Alexandru, C.V., Shimagaki, J., Visaggio, C.A., Canfora, G., Gall, H.C., What would users change in my app? summarizing app reviews for recommending software changes. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE, pp. 499–510.

[9]

Di Sorbo A., Panichella S., Alexandru C.V., Visaggio C.A., Canfora G., SURF: summarizer of user reviews feedback, in: Proceedings of the IEEE/ACM 39th International Conference on Software Engineering Companion, IEEE, 2017, pp. 55–58.

[10]

Dong, L., Xu, S., Xu, B., 2018. Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition. In: Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5884–5888.

[11]

Fang S., Tan Y.-S., Zhang T., Liu Y., Self-attention networks for code search, Inf. Softw. Technol. 134 (2021).

[12]

Fernandes P., Allamanis M., Brockschmidt M., Structured neural summarization, 2021, arXiv:1811.01824.

[13]

Gage P., A new algorithm for data compression, C Users J. 12 (1994) 23–38.

[14]

Gao, Y., Wang, W., Herold, C., Yang, Z., Ney, H., 2020. Towards a better understanding of label smoothing in neural machine translation. In: Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, AACL/IJCNLP, pp. 212–223.

[15]

Gao, C., Zeng, J., Lo, D., Lin, C.-Y., Lyu, M.R., King, I., 2018. INFAR: Insight extraction from app reviews. In: Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE, pp. 904–907.

[16]

Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N., 2017. Convolutional sequence to sequence learning. In: Proceedings of the 34th International Conference on Machine Learning, ICML, pp. 1243–1252.

[17]

Gehrmann S., Deng Y., Rush A.M., Bottom-up abstractive summarization, 2018, arXiv:1808.10792.

[18]

Gousios, G., Pinzger, M., Deursen, A.v., 2014. An exploratory study of the pull-based software development model. In: Proceedings of the 36th International Conference on Software Engineering, ICSE, pp. 345–355.

[19]

Gousios, G., Storey, M.-A., Bacchelli, A., 2016. Work practices and challenges in pull-based development: the contributor’s perspective. In: Proceedings of the IEEE/ACM 38th International Conference on Software Engineering, ICSE, pp. 285–296.

[20]

Gousios, G., Zaidman, A., Storey, M.-A., Van Deursen, A., 2015. Work practices and challenges in pull-based development: the integrator’s perspective. In: Proceedings of the IEEE/ACM 37th IEEE International Conference on Software Engineering, ICSE, pp. 358–368.

[21]

Gu, J., Lu, Z., Li, H., Li, V.O.K., 2016. Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL, pp. 1631–1640.

[22]

Gu X., Zhang H., Kim S., Deep code search, in: Proceedings of the IEEE/ACM 40th International Conference on Software Engineering, ICSE, IEEE, 2018, pp. 933–944.

[23]

Hochreiter S., Schmidhuber J., Long short-term memory, Neural Comput. 9 (8) (1997) 1735–1780.

Digital Library

[24]

Hu, X., Li, G., Xia, X., Lo, D., Jin, Z., 2018. Deep code comment generation. In: Proceedings of the IEEE/ACM 26th International Conference on Program Comprehension, ICPC, pp. 200–20010.

[25]

Huang, Y., Zheng, Q., Chen, X., Xiong, Y., Liu, Z., Luo, X., 2017. Mining version control system for automatically generating commit comment. In: Proceedings of the 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 414–423.

[26]

Iyer, S., Konstas, I., Cheung, A., Zettlemoyer, L., 2016. Summarizing source code using a neural attention model. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL, pp. 2073–2083.

[27]

Jiang, S., Armaly, A., McMillan, C., 2017. Automatically generating commit messages from diffs using neural machine translation. In: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering. ASE, pp. 135–146.

[28]

Kalchbrenner, N., Blunsom, P., 2013. Recurrent continuous translation models. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp. 1700–1709.

[29]

Karampatsis, R.-M., Babii, H., Robbes, R., Sutton, C., Janes, A., 2020. Big code!=big vocabulary: Open-vocabulary models for source code. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering, ICSE, pp. 1073–1085.

[30]

Kingma D.P., Ba J., Adam: A method for stochastic optimization, 2017, arXiv:1412.6980.

[31]

Lan Z., Chen M., Goodman S., Gimpel K., Sharma P., Soricut R., ALBERT: A lite BERT for self-supervised learning of language representations, 2020, arXiv:1909.11942.

[32]

Lebanoff L., Song K., Dernoncourt F., Kim D.S., Kim S., Chang W., Liu F., Scoring sentence singletons and pairs for abstractive summarization, 2019, arXiv:1906.00077.

[33]

LeClair, A., Jiang, S., McMillan, C., 2019. A neural model for generating natural language summaries of program subroutines. In: Proceedings of the IEEE/ACM 41st International Conference on Software Engineering, ICSE, pp. 795–806.

[34]

LeClair A., McMillan C., Recommendations for datasets for source code summarization, 2019, arXiv:1904.02660.

[35]

LeCun Y., Bengio Y., Hinton G., Deep learning, Nature 521 (7553) (2015) 436–444.

[36]

Lee J., Yoon W., Kim S., Kim D., Kim S., So C.H., Kang J., BioBERT: a pre-trained biomedical language representation model for biomedical text mining, Bioinformatics 36 (4) (2020) 1234–1240.

[37]

Li, P., Lam, W., Bing, L., Wang, Z., 2017. Deep recurrent generative decoder for abstractive text summarization. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2091–2100.

[38]

Li H., Zheng Y., Ren P., Dual-channel attention model for text sentiment analysis, Int. J. Perform. Eng. 15 (3) (2019) 834–841.

[39]

Lin, C.-Y., 2004. Rouge: A package for automatic evaluation of summaries. In: Proceedings of the Text Summarization Branches Out, pp. 74–81.

[40]

Liu S., Chen Y., Xie X., Siow J., Liu Y., Retrieval-augmented generation for code summarization via hybrid GNN, 2021, arXiv:2006.05405.

[41]

Liu, Y., Lapata, M., Text summarization with pretrained encoders. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP, pp. 3728–3738.

[42]

Liu Y., Ott M., Goyal N., Du J., Joshi M., Chen D., Levy O., Lewis M., Zettlemoyer L., Stoyanov V., RoBERTa: A robustly optimized BERT pretraining approach, 2019, arXiv:1907.11692.

[43]

Liu, Z., Xia, X., Treude, C., Lo, D., Li, S., 2019b. Automatic generation of pull request descriptions. In: Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering, ASE, pp. 176–188.

[44]

Loyola, P., Marrese-Taylor, E., Matsuo, Y., 2017. A neural architecture for generating natural language descriptions from source code changes. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL, pp. 287–292.

[45]

Luong, T., Pham, H., Manning, C.D., 2015. Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp. 1412–1421.

[46]

Lutellier, T., Pham, H.V., Pang, L., Li, Y., Wei, M., Tan, L., 2020. CoCoNuT: combining context-aware neural translation models using ensemble for program repair. In: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 101–114.

[47]

Moreno, L., Bavota, G., Di Penta, M., Oliveto, R., Marcus, A., Canfora, G., Automatic generation of release notes, Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE, pp. 484–495.

[48]

Moreno L., Bavota G., Di Penta M., Oliveto R., Marcus A., Canfora G., ARENA: an approach for the automated generation of release notes, IEEE Trans. Softw. Eng. (TSE) 43 (2016) 106–127.

[49]

Müller, R., Kornblith, S., Hinton, G.E., 2019. When does label smoothing help? In: Proceedings of the 33rd Conference on Neural Information Processing Systems, NIPS, pp. 4694–4703.

[50]

Nie L.Y., Gao C., Zhong Z., Lam W., Liu Y., Xu Z., Contextualized code representation learning for commit message generation, 2020, arXiv:2007.06934.

[51]

Papineni, K., Roukos, S., Ward, T., Zhu, W.-J., 2002. Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318.

[52]

Rahman, M.M., Roy, C.K., 2014. An insight into the pull requests of github. In: Proceedings of the 11th Working Conference on Mining Software Repositories, MSR, pp. 364–367.

[53]

Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., Goel, V., 2017. Self-critical sequence training for image captioning. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 7008–7024.

[54]

See, A., Liu, P.J., Manning, C.D., 2017. Get to the point: Summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL, pp. 1073–1083.

[55]

Sennrich, R., Haddow, B., Birch, A., 2016. Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL, pp. 1715–1725.

[56]

Shaw P., Uszkoreit J., Vaswani A., Self-attention with relative position representations, 2018, arXiv:1803.02155.

[57]

Stapleton, S., Gambhir, Y., LeClair, A., Eberhart, Z., Weimer, W., Leach, K., Huang, Y., 2020. A Human study of comprehension and code summarization. In: Proceedings of the 28th International Conference on Program Comprehension, pp. 2–13.

[58]

Sutskever, I., Vinyals, O., Le, Q.V., 2014. Sequence to sequence learning with neural networks. In: Proceedings of the 28th Conference on Neural Information Processing Systems, NIPS, pp. 3104–3112.

[59]

Tsay, J., Dabbish, L., Herbsleb, J., 2014. Influence of social and technical factors for evaluating contribution in GitHub. In: Proceedings of the 36th International Conference on Software Engineering, ICSE, pp. 356–366.

[60]

Van Der Veen, E., Gousios, G., Zaidman, A., 2015. Automatically prioritizing pull requests. In: Proceedings of the IEEE/ACM 12th Working Conference on Mining Software Repositories, MSR, pp. 357–361.

[61]

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I., 2017. Attention is all you need. In: Proceedings of the 31st Conference on Neural Information Processing Systems, NIPS, pp. 5998–6008.

[62]

Veličković P., Cucurull G., Casanova A., Romero A., Liò P., Bengio Y., Graph attention networks, 2018, arXiv:1710.10903.

[63]

Vinyals, O., Fortunato, M., Jaitly, N., 2015. Pointer networks. In: Proceedings of the 29th Conference on Neural Information Processing Systems, NIPS, pp. 2692–2700.

[64]

Wang, Y., Chen, T., Xu, H., Ding, S., Lv, H., Shao, Y., Peng, N., Xie, L., Watanabe, S., Khudanpur, S., 2019. Espresso: A fast end-to-end neural speech recognition toolkit. In: Proceedings of the 2019 IEEE Automatic Speech Recognition and Understanding Workshop, pp. 136–143.

[65]

Wei, B., Li, Y., Li, G., Xia, X., Jin, Z., 2020. Retrieve and refine: exemplar-based neural comment generation. In: Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, ASE, pp. 349–360.

[66]

Wu Y., Schuster M., Chen Z., Le Q.V., Norouzi M., Macherey W., Krikun M., Cao Y., Gao Q., Macherey K., Klingner J., Shah A., Johnson M., Liu X., Kaiser Ł., Gouws S., Kato Y., Kudo T., Kazawa H., Stevens K., Kurian G., Patil N., Wang W., Young C., Smith J., Riesa J., Rudnick A., Vinyals O., Corrado G., Hughes M., Dean J., Google’s neural machine translation system: Bridging the gap between human and machine translation, 2016, arXiv:1609.08144.

[67]

Xu, M., Wong, D.F., Yang, B., Zhang, Y., Chao, L.S., 2019a. Leveraging local and global patterns for self-attention networks. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, ACL, pp. 3069–3075.

[68]

Xu, S., Yao, Y., Xu, F., Gu, T., Tong, H., Lu, J., 2019b. Commit message generation for source code changes. In: Proceedings of the 2019 28th International Joint Conference on Artificial Intelligence, IJCAI, pp. 3975–3981.

[69]

Yang, B., Tu, Z., Wong, D.F., Meng, F., Chao, L.S., Zhang, T., 2018. Modeling localness for self-attention networks. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp. 4449–4458.

[70]

Yu A.W., Dohan D., Luong M.-T., Zhao R., Chen K., Norouzi M., Le Q.V., QANet: Combining local convolution with global self-attention for reading comprehension, 2018, arXiv:1804.09541.

[71]

Yu, X., Huang, Q., Wang, Z., Feng, Y., Zhao, D., 2020. Towards context-aware code comment generation. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp. 3938–3947.

[72]

Yu, Y., Wang, H., Filkov, V., Devanbu, P., Vasilescu, B., 2015. Wait for it: Determinants of pull request evaluation latency on github. In: Proceedings of the IEEE/ACM 12th Working Conference on Mining Software Repositories, MSR, pp. 367–371.

[73]

Yu, Y., Wang, H., Yin, G., Ling, C.X., 2014. Reviewer recommender of pull-requests in github. In: Proceedings of the 2014 IEEE International Conference on Software Maintenance and Evolution, pp. 609–612.

[74]

Zhang, T., Chen, J., Jiang, H., Luo, X., Xia, X., 2017. Bug report enrichment with application of automated fixer recommendation. In: Proceedings of the IEEE/ACM 25th International Conference on Program Comprehension, ICPC, pp. 230–240.

[75]

Zhang, H., Goodfellow, I., Metaxas, D., Odena, A., 2019. Self-attention generative adversarial networks. In: Proceedings of the 36th International Conference on Machine Learning, ICML, pp. 7354–7363.

[76]

Zhao C., Wang J., Service recommendation model based on rating matrix and context-embedded lstm, Int. J. Perform. Eng. 15 (2019) 2432–2431.

Cited By

Tufano RMastropaolo APepe FDabic ODi Penta MBavota GSpinellis DConstantinou EBacchelli A(2024)Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based StudyProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644918(571-583)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643991.3644918
Xiao THata HTreude CMatsumoto K(2024)Generative AI for Pull Request Descriptions: Adoption, Impact, and Developer InterventionsProceedings of the ACM on Software Engineering10.1145/36437731:FSE(1043-1065)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643773
Fang SZhang TTan YJiang HXia XSun XGrundy JPollock LPenta M(2023)RepresentThemAll: A Universal Learning Representation of Bug ReportsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00060(602-614)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00060
Show More Cited By

Index Terms

PRHAN: Automated Pull Request Description Generation Based on Hybrid Attention Network

Index terms have been assigned to the content through auto-classification.

Recommendations

TransBoNet: Learning camera localization with Transformer Bottleneck and Attention
Abstract
6DoF camera localization is an important component of autonomous driving and navigation. Deep learning has achieved impressive results in localization, but its robustness in dynamic environments has not been adequately addressed. In this paper, ...
Highlights
- We propose a novel Transformer Bottleneck block with self-attention and channel attention to overcome the limitations of convolution. This coupling allows them to be optimized in a mutually reinforcing manner, significantly improving fine-...
Hybrid Attention for Chinese Character-Level Neural Machine Translation
Abstract
This paper proposes a novel character-level neural machine translation model which can effectively improve the Neural Machine Translation (NMT) by fusing word and character attention information. In our work, the bidirectional Gated Recurrent ...
Open-vocabulary models for source code
ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings

Statistical language modeling techniques have successfully been applied to large source code corpora, yielding a variety of new software development tools, such as tools for code suggestion, improving readability, and API migration. A major issue with ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Journal of Systems and Software

Journal of Systems and Software Volume 185, Issue C

Mar 2022

145 pages

ISSN:0164-1212

Issue’s Table of Contents

Elsevier Inc.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 March 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tufano RMastropaolo APepe FDabic ODi Penta MBavota GSpinellis DConstantinou EBacchelli A(2024)Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based StudyProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644918(571-583)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643991.3644918
Xiao THata HTreude CMatsumoto K(2024)Generative AI for Pull Request Descriptions: Adoption, Impact, and Developer InterventionsProceedings of the ACM on Software Engineering10.1145/36437731:FSE(1043-1065)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643773
Fang SZhang TTan YJiang HXia XSun XGrundy JPollock LPenta M(2023)RepresentThemAll: A Universal Learning Representation of Bug ReportsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00060(602-614)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00060
Karmakar SCodabux ZVidoni M(2022)An Experience Report on Technical Debt in Pull Requests: Challenges and Lessons LearnedProceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3544902.3546637(295-300)Online publication date: 19-Sep-2022
https://dl.acm.org/doi/10.1145/3544902.3546637

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents