Document Open Access Logo

Type Regression Testing to Detect Breaking Changes in Node.js Libraries

Authors Gianluca Mezzetti, Anders Møller, Martin Toldam Torp

PDF
Thumbnail PDF

File

LIPIcs.ECOOP.2018.7.pdf
  • Filesize: 0.72 MB
  • 24 pages

Document Identifiers

Author Details

Gianluca Mezzetti
  • Aarhus University, Denmark
Anders Møller
  • Aarhus University, Denmark
Martin Toldam Torp
  • Aarhus University, Denmark

Funding

This work was supported by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No 647544).

Cite AsGet BibTex

Gianluca Mezzetti, Anders Møller, and Martin Toldam Torp. Type Regression Testing to Detect Breaking Changes in Node.js Libraries. In 32nd European Conference on Object-Oriented Programming (ECOOP 2018). Leibniz International Proceedings in Informatics (LIPIcs), Volume 109, pp. 7:1-7:24, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2018)
https://doi.org/10.4230/LIPIcs.ECOOP.2018.7

Abstract

The npm repository contains JavaScript libraries that are used by millions of software developers. Its semantic versioning system relies on the ability to distinguish between breaking and non-breaking changes when libraries are updated. However, the dynamic nature of JavaScript often causes unintended breaking changes to be detected too late, which undermines the robustness of the applications. We present a novel technique, type regression testing, to automatically determine whether an update of a library implementation affects the types of its public interface, according to how the library is being used by other npm packages. By leveraging available test suites of clients, type regression testing uses a dynamic analysis to learn models of the library interface. Comparing the models before and after an update effectively amplifies the existing tests by revealing changes that may affect the clients. Experimental results on 12 widely used libraries show that the technique can identify type-related breaking changes with high accuracy. It fully automatically classifies at least 90% of the updates correctly as either major or as minor or patch, and it detects 26 breaking changes among the minor and patch updates.

Subject Classification

ACM Subject Classification
  • Software and its engineering → Software libraries and repositories
Keywords
  • JavaScript
  • semantic versioning
  • dynamic analysis

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. Christoffer Quist Adamsen, Gianluca Mezzetti, and Anders Møller. Analyzing test completeness for dynamic languages. In Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18-20, 2016, pages 142-153, 2016. URL: http://dx.doi.org/10.1145/2931037.2931059.
  2. Jong-hoon (David) An, Avik Chaudhuri, Jeffrey S. Foster, and Michael Hicks. Dynamic inference of static types for Ruby. In Proceedings of the 38th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2011, Austin, TX, USA, January 26-28, 2011, pages 459-472, 2011. URL: http://dx.doi.org/10.1145/1926385.1926437.
  3. Esben Andreasen, Colin S. Gordon, Satish Chandra, Manu Sridharan, Frank Tip, and Koushik Sen. Trace typing: An approach for evaluating retrofitted type systems. In 30th European Conference on Object-Oriented Programming, ECOOP 2016, July 18-22, 2016, Rome, Italy, pages 1:1-1:26, 2016. URL: http://dx.doi.org/10.4230/LIPIcs.ECOOP.2016.1.
  4. Esben Andreasen, Anders Møller, Liang Gong, Michael Pradel, Marija Selakovic, Koushik Sen, and Cristian-Alexandru Staicu. A survey of dynamic analysis and test generation for JavaScript. ACM Computing Surveys, 50(5):66:1-66:36, 2017. Google Scholar
  5. Thomas H. Austin, Tim Disney, Alan Jeffrey, and Cormac Flanagan. Dynamic information flow analysis for featherweight JavaScript. Technical Report UCSC-SOE-11-19, UC Santa Cruz, 2011. Google Scholar
  6. Martin Bodin, Arthur Charguéraud, Daniele Filaretti, Philippa Gardner, Sergio Maffeis, Daiva Naudziuniene, Alan Schmitt, and Gareth Smith. A trusted mechanised JavaSript specification. In The 41st Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL '14, San Diego, CA, USA, January 20-21, 2014, pages 87-100. ACM, 2014. Google Scholar
  7. Christopher Bogart, Christian Kästner, and James D. Herbsleb. When it breaks, it breaks: How ecosystem developers reason about the stability of dependencies. In 30th IEEE/ACM International Conference on Automated Software Engineering Workshops, ASE Workshops 2015, Lincoln, NE, USA, November 9-13, 2015, pages 86-89, 2015. URL: http://dx.doi.org/10.1109/ASEW.2015.21.
  8. Christopher Bogart, Christian Kästner, James D. Herbsleb, and Ferdian Thung. How to break an API: cost negotiation and community values in three software ecosystems. In Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, Seattle, WA, USA, November 13-18, 2016, pages 109-120, 2016. URL: http://dx.doi.org/10.1145/2950290.2950325.
  9. Erik Derr, Sven Bugiel, Sascha Fahl, Yasemin Acar, and Michael Backes. Keep me updated: An empirical study of third-party library updatability on Android. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS 2017, Dallas, TX, USA, October 30 - November 03, 2017, pages 2187-2200, 2017. Google Scholar
  10. Amin Milani Fard and Ali Mesbah. JavaScript: The (un)covered parts. In 2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, Tokyo, Japan, March 13-17, 2017, pages 230-240. IEEE Computer Society, 2017. Google Scholar
  11. Cormac Flanagan, Amr Sabry, Bruce F. Duba, and Matthias Felleisen. The essence of compiling with continuations. In Proceedings of the ACM SIGPLAN'93 Conference on Programming Language Design and Implementation (PLDI), Albuquerque, New Mexico, USA, June 23-25, 1993, pages 237-247, 1993. URL: http://dx.doi.org/10.1145/155090.155113.
  12. Ira R. Forman, Michael H. Conner, Scott Danforth, and Larry K. Raper. Release-to-release binary compatibility in SOM. In OOPSLA'95, Proceedings of the Tenth Annual Conference on Object-Oriented Programming Systems, Languages, and Applications, Austin, Texas, USA, October 15-19, 1995, pages 426-438. ACM, 1995. Google Scholar
  13. James Gosling, Bill Joy, Guy L. Steele, Gilad Bracha, and Alex Buckley. The Java Language Specification, Java SE 8 Edition. Addison-Wesley Professional, 1st edition, 2014. Google Scholar
  14. Arjun Guha, Claudiu Saftoiu, and Shriram Krishnamurthi. The essence of JavaScript. In ECOOP 2010 - Object-Oriented Programming, 24th European Conference, Maribor, Slovenia, June 21-25, 2010. Proceedings, pages 126-150, 2010. Google Scholar
  15. Atsushi Igarashi and Hideshi Nagira. Union types for object-oriented programming. Journal of Object Technology, 6(2):47-68, 2007. OOPS Track at the 21st ACM Symposium on Applied Computing, SAC 2006. URL: http://dx.doi.org/10.5381/jot.2007.6.2.a3.
  16. Kamil Jezek, Jens Dietrich, and Premek Brada. How Java APIs break - an empirical study. Information & Software Technology, 65:129-146, 2015. URL: http://dx.doi.org/10.1016/j.infsof.2015.02.014.
  17. Matthias Keil, Sankha Narayan Guria, Andreas Schlegel, Manuel Geffken, and Peter Thiemann. Transparent object proxies in JavaScript. In 29th European Conference on Object-Oriented Programming, ECOOP 2015, July 5-10, 2015, Prague, Czech Republic, pages 149-173, 2015. URL: http://dx.doi.org/10.4230/LIPIcs.ECOOP.2015.149.
  18. Riivo Kikas, Georgios Gousios, Marlon Dumas, and Dietmar Pfahl. Structure and evolution of package dependency networks. In Proceedings of the 14th International Conference on Mining Software Repositories, MSR 2017, Buenos Aires, Argentina, May 20-28, 2017, pages 102-112. IEEE Computer Society, 2017. Google Scholar
  19. Raula Gaikovina Kula, Ali Ouni, Daniel M. Germán, and Katsuro Inoue. On the impact of micro-packages: An empirical study of the npm JavaScript ecosystem. CoRR, abs/1709.04638, 2017. URL: http://arxiv.org/abs/1709.04638.
  20. Tobias Lauinger, Abdelberi Chaabane, Sajjad Arshad, William Robertson, Christo Wilson, and Engin Kirda. Thou shalt not depend on me: Analysing the use of outdated JavaScript libraries on the web. Proceedings of Network and Distributed System Security Symposium (NDSS), 2017. Google Scholar
  21. Barbara Liskov and Jeannette M. Wing. A behavioral notion of subtyping. ACM Trans. Program. Lang. Syst., 16(6):1811-1841, 1994. URL: http://dx.doi.org/10.1145/197320.197383.
  22. Samim Mirhosseini and Chris Parnin. Can automated pull requests encourage software developers to upgrade out-of-date dependencies? In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, ASE 2017, Urbana, IL, USA, October 30 - November 03, 2017, pages 84-94, 2017. URL: http://dx.doi.org/10.1109/ASE.2017.8115621.
  23. Andrey V. Ponomarenko and Vladimir V. Rubanov. Backward compatibility of software interfaces: Steps towards automatic verification. Programming and Computer Software, 38(5):257-267, 2012. Google Scholar
  24. Michael Pradel, Parker Schuh, and Koushik Sen. TypeDevil: Dynamic type inconsistency analysis for JavaScript. In 37th IEEE/ACM International Conference on Software Engineering, ICSE 2015, Florence, Italy, May 16-24, 2015, Volume 1, pages 314-324, 2015. URL: http://dx.doi.org/10.1109/ICSE.2015.51.
  25. Tom Preston-Werner. Semantic versioning 2.0.0. http://semver.org/.
  26. Steven Raemaekers, Arie Van Deursen, and Joost Visser. Semantic versioning versus breaking changes: A study of the Maven repository. Proceedings - 2014 14th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2014, pages 215-224, 2014. URL: http://dx.doi.org/10.1109/SCAM.2014.30.
  27. Claudiu Saftoiu, Arjun Guha, and Shriram Krishnamurthi. Runtime type-discovery for JavaScript. Technical Report Brown University CS-10-05, 2010. Google Scholar
  28. Marina Sokolova and Guy Lapalme. A systematic analysis of performance measures for classification tasks. Inf. Process. Manage., 45(4):427-437, 2009. Google Scholar
  29. Sofia Visa, Brian Ramsay, Anca L. Ralescu, and Esther van der Knaap. Confusion matrix-based feature selection. In Proceedings of the 22nd Midwest Artificial Intelligence and Cognitive Science Conference 2011, Cincinnati, Ohio, USA, April 16-17, 2011, pages 120-127, 2011. URL: http://ceur-ws.org/Vol-710/paper37.pdf.
  30. Erik Wittern, Philippe Suter, and Shriram Rajagopalan. A look at the dynamics of the JavaScript package ecosystem. In Proceedings of the 13th International Conference on Mining Software Repositories, MSR 2016, Austin, TX, USA, May 14-22, 2016, pages 351-361, 2016. URL: http://dx.doi.org/10.1145/2901739.2901743.
  31. Ahmed Zerouali, Eleni Constantinou, Tom Mens, Gregorio Robles, and Jesús M. González-Barahona. An empirical analysis of technical lag in npm package dependencies. In New Opportunities for Software Reuse - 17th International Conference, ICSR 2018, Madrid, Spain, May 21-23, 2018, Proceedings, volume 10826 of Lecture Notes in Computer Science, pages 95-110. Springer, 2018. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact