research-article

Fragment-Based Test Generation for Web Apps

Authors:

Rahul Krishna Yandrapally,

Ali MesbahAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 49, Issue 3

Pages 1086 - 1101

https://doi.org/10.1109/TSE.2022.3171295

Published: 01 March 2023 Publication History

Abstract

Automated model-based test generation presents a viable alternative to the costly manual test creation currently employed for regression testing of web apps. However, existing model inference techniques rely on threshold-based whole-page comparison to establish state equivalence, which cannot reliably identify near-duplicate web pages in modern web apps. Consequently, existing techniques produce inadequate models for dynamic web apps, and fragile test oracles, rendering the generated regression test suites ineffective. We propose a model-based test generation technique, <sc>FragGen</sc>, that eliminates the need for thresholds, by employing a novel state abstraction based on page fragmentation to establish state equivalence. <sc>FragGen</sc> also uses fine-grained page fragment analysis to diversify state exploration and generate reliable test oracles. Our evaluation shows that <sc>FragGen</sc> outperforms existing whole-page techniques by detecting more near-duplicates, inferring better web app models and generating test suites that are better suited for regression testing. On a dataset of 86,165 state-pairs, <sc>FragGen</sc> detected 123% more near-duplicates on average compared to whole-page techniques. The crawl models inferred by <sc>FragGen</sc> have 62% more precision and 70% more recall on average. <sc>FragGen</sc> also generates reliable regression test suites with test actions that have nearly 100% success rate on the same version of the web app even if the execution environment is varied. The test oracles generated by <sc>FragGen</sc> can detect 98.7% of the visible changes in web pages while being highly robust, making them suitable for regression testing.

References

[1]

S. Thummalapenta, K. V. Lakshmi, S. Sinha, N. Sinha, and S. Chandra, “Guided test generation for web applications,” in Proc. 35th Int. Conf. Softw. Eng., 2013, pp. 162–171.

[2]

D. Roest, A. Mesbah, and A. V. Deursen, “Regression testing ajax applications: Coping with dynamism,” in Proc. Int. Conf. Softw. Testing, Verification Validation, 2010, pp. 127–136.

[3]

A. M. Memon and M. L. Soffa, “Regression testing of GUIs,” SIGSOFT Softw. Eng. Notes, vol. 28, no. 5, pp. 118–127, Sep. 2003.

Digital Library

[4]

M. Grechanik, Q. Xie, and C. Fu, “Maintaining and evolving GUI-directed test scripts,” in Proc. 31st Int. Conf. Softw. Eng., 2009, pp. 408–418.

[5]

L. Christophe, R. Stevens, C. D. Roover, and W. D. Meuter, “Prevalence and maintenance of automated functional tests for web applications,” in Proc. IEEE Int. Conf. Softw. Maintenance Evol., 2014, pp. 141–150.

[6]

M. Biagiola, A. Stocco, F. Ricca, and P. Tonella, “Diversity-based web test generation,” in Proc. 27th ACM Joint Meeting Eur. Softw. Eng. Conf. Symp. Found. Softw. Eng., 2019, pp. 142–153.

[7]

A. Mesbah, A. van Deursen, and D. Roest, “Invariant-based automatic testing of modern web applications,” IEEE Trans. Softw. Eng., vol. 38, no. 1, pp. 35–53, Jan./Feb. 2012.

Digital Library

[8]

S. Mirshokraie and A. Mesbah, “JSART: JavaScript assertion-based regression testing,” in Proc. Int. Conf. Web Eng., 2012, pp. 238–252.

[9]

A. Mesbah, A. van Deursen, and S. Lenselink, “Crawling ajax-based web applications through dynamic analysis of user interface state changes,” ACM Trans. Web, vol. 6, no. 1, pp. 3:1–3:30, 2012.

Digital Library

[10]

R. K. Yandrapally, A. Stocco, and A. Mesbah, “Near-duplicate detection in web app model inference,” in Proc. ACM/IEEE Int. Conf. Softw. Eng., 2020, Art. no.

[11]

M. Henzinger, “Finding near-duplicate web pages: A large-scale evaluation of algorithms,” in Proc. 29th Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2006, pp. 284–291.

[12]

M. Leotta, A. Stocco, F. Ricca, and P. Tonella, “Using multi-locators to increase the robustness of web test cases,” in Proc. 8th Int. Conf. Softw. Testing, Verification Validation, 2015, pp. 1–10.

[13]

anon, FragGen VM, 2020. [Online]. Available: https://doi.org/10.5281/zenodo.4007539

[14]

PHP AddressBook, Simple, web-based address & phone book, Accessed: Oct.01, 2018, 2015. [Online]. Available: http://sourceforge.net/projects/php-addressbook

[15]

A. Mesbah and A. van Deursen, “Invariant-based automatic testing of AJAX user interfaces,” in Proc. 31st Int. Conf. Softw. Eng., 2009, pp. 210–220.

[16]

G. A. Di Lucca, M. Di Penta, A. R. Fasolino, and P. Granato, “Clone analysis in the web era: An approach to identify cloned web pages,” in Proc. Int. Workshop Empir. Stud. Softw. Maintenance, 2001, pp. 107–113.

[17]

S. R. Choudhary, M. R. Prasad, and A. Orso, “CrossCheck: Combining crawling and differencing to better detect cross-browser incompatibilities in web applications,” in Proc. IEEE 5th Int. Conf. Softw. Testing, Verification Validation, 2012, pp. 171–180.

[18]

S. Roy Choudhary, M. R. Prasad, and A. Orso, “X-PERT: Accurate identification of cross-browser issues in web applications,” in Proc. Int. Conf. Softw. Eng., 2013, pp. 702–711.

[19]

S. Mahajan and W. G. J. Halfond, “Detection and localization of HTML presentation failures using computer vision-based techniques,” in Proc. 2015 IEEE 8th Int. Conf. Softw. Testing, Verification Validation, ser. ICST ’15, 2015, pp. 1–10.

[20]

S. Mahajan and W. G. Halfond, “Finding HTML presentation failures using image comparison techniques,” in Proc. 29th ACM/IEEE Int. Conf. Automated Softw. Eng., 2014, pp. 91–96.

[21]

M. Pawlik and N. Augsten, “Efficient computation of the tree edit distance,” ACM Trans. Database Syst., vol. 40, no. 1, pp. 3:1–3:40, Mar. 2015.

Digital Library

[22]

V. Levenshtein, “Binary codes capable of correcting deletions, insertions and reversals,” Sov. Phys. Doklady, vol. 10, 1966, Art. no.

[23]

J. Oliver, C. Cheng, and Y. Chen, “TLSH - A locality sensitive hash,” in Proc. 4th Cybercrime Trustworthy Comput. Workshop, 2013, pp. 7–13.

[24]

M. S. Charikar, “Similarity estimation techniques from rounding algorithms,” in Proc 34th Annu. ACM Symp. Theory Comput., 2002, pp. 380–388.

[25]

M. J. Swain and D.H. Ballard, “Indexing via color histograms” in Active Perception and Robot Vision, A. K. Sood and H. Wechsler, Eds., Berlin, Germany: Springer, 1992, pp. 261–273.

[26]

B. Yang, F. Gu, and X. Niu, “Block mean value based image perceptual hashing,” in Proc. Int. Conf. Intell. Inf. Hiding Multimedia, 2006, pp. 167–172.

[27]

C. Zauner, “Implementation and benchmarking of perceptual image hash functions,” Ph.D. dissertation, 2010.

[28]

H. Yee, S. Pattanaik, and D. P. Greenberg, “Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments,” ACM Trans. Graph., vol. 20, no. 1, pp. 39–65, Jan. 2001.

Digital Library

[29]

D. G. Lowe, “Object recognition from local scale-invariant features,” in Proc. 7th IEEE Int. Conf. Comput. Vis., 1999, pp. 1150–1157.

[30]

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004.

Digital Library

[31]

D. Cai, S. Yu, J.-R. Wen, and W.-Y. Ma, “Vips: A vision-based page segmentation algorithm,” Tech. Rep. MSR-TR-2003-79, p. 28, Nov. 2003. [Online]. Available: https://www.microsoft.com/en-us/research/publication/vips-a-vision-based-page-segmentation-algorithm/

[32]

J. Kiesel, L. Meyer, F. Kneist, B. Stein, and M. Potthast, “An empirical comparison of web page segmentation algorithms,” in Proc. Eur. Conf. Inf. Retrieval, 2021, pp. 62–74.

[33]

“Document object model (DOM).” Accessed: Jul. 07, 2022. [Online]. Available: https://en.wikipedia.org/wiki/Document_Object_Model

[34]

M. Pawlik and N. Augsten, “Tree edit distance: Robust and memory-efficient,” Inf. Syst., vol. 56, pp. 157–173, 2016.

Digital Library

[35]

“XPath XML path language.” Accessed: Jul. 01, 2022. [Online]. Available: https://en.wikipedia.org/wiki/XPath#Abbreviated_syntax

[36]

A. Torsel, “Automated test case generation for web applications from a domain specific model,” in Proc. IEEE 35th Annu. Comput. Softw. Appl. Conf. Workshops, 2011, pp. 137–142.

[37]

E. T. Barr, M. Harman, P. McMinn, M. Shahbaz, and S. Yoo, “The oracle problem in software testing: A survey,” IEEE Trans. Softw. Eng., vol. 41, no. 5, pp. 507–525, May 2015.

Digital Library

[38]

T. Popela, “Implementace algoritmu pro vizuální segmentaci www stránek,” Master's thesis, Faculty Inf. Technol., Brno Univ. Technol., Brno, Czechia, 2012. [Online]. Available: https://www.fit.vut.cz/study/thesis/14163/

[39]

Angular version of the spring PetClinic web application, 2018. [Online]. Available: https://github.com/spring-petclinic/spring-petclinic-angular

[40]

PHP Password Manager, 2018. [Online]. Available: https://github.com/pklink/ppma

[41]

DimeShift: Easiest way to track your expenses, 2018. [Online]. Available: https://github.com/jeka-kiselyov/dimeshift

[42]

Claroline, Open source learning management system, 2015. [Online]. Available: https://sourceforge.net/projects/claroline/

[43]

Phoenix: Trello tribute done in elixir, phoenix framework, react and redux, 2017. [Online]. Available: https://github.com/jesuvalan/phoenix-trello

[44]

Pagekit: Modular and lightweight CMS., 2018. [Online]. Available: https://github.com/pagekit/pagekit

[45]

Mantis Bug Tracker, 2018. [Online]. Available: https://github.com/mantisbt/mantisbt

[46]

N. D. Study, Near duplicate study crawls, Sep.2019. [Online]. Available: https://doi.org/10.5281/zenodo.3385377

[47]

A. Stocco, M. Leotta, F. Ricca, and P. Tonella, “Clustering-aided page object generation for web testing,” in Proc. 16th Int. Conf. Web Eng., 2016, pp. 132–151.

[48]

A. Stocco, M. Leotta, F. Ricca, and P. Tonella, “APOGEN: Automatic page object generator for web testing,” Softw. Qual. J., vol. 25, no. 3, pp. 1007–1039, Sep. 2017.

Digital Library

[49]

M. Biagiola, A. Stocco, A. Mesbah, F. Ricca, and P. Tonella, “Web test dependency detection,” in Proc. 27th ACM Joint Eur. Softw. Eng. Conf. Symp. Found. Softw. Eng., 2019, Art. no.

[50]

J. Snoek, H. Larochelle, and R. P. Adams, “Practical Bayesian optimization of machine learning algorithms,” in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 2951–2959. [Online]. Available: http://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf

[51]

MacOS Mojave, 2020. [Online]. Available: https://en.wikipedia.org/wiki/MacOS_Mojave

[52]

RedHat Linux, 2018. [Online]. Available: https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#RHEL_7

[53]

K. Nishiura, Y. Maezawa, H. Washizaki, and S. Honiden, “Mutation analysis for javascript web applications testing,” in Proc. Int. Conf. Softw. Eng. Knowl. Eng., 2013, pp. 159–165.

[54]

S. Mirshokraie, A. Mesbah, and K. Pattabiraman, “Guided mutation testing for javascript web applications,” IEEE Trans. Softw. Eng., vol. 41, no. 5, pp. 429–444, May 2015.

Digital Library

[55]

U. Praphamontripong and J. Offutt, “Applying mutation testing to web applications,” in Proc. 3rd Int. Conf. Softw. Testing, Verification, Validation Workshops, 2010, pp. 132–141.

[56]

M. Linares-Vásquezet al., “Enabling mutation testing for android apps,” in Proc. 11th Joint Meeting Found. Softw. Eng., 2017, pp. 233–244.

[57]

R. A. P. Oliveira, E. AlÃ©groth, Z. Gao, and A. Memon, “Definition and evaluation of mutation operators for GUI-level mutation analysis,” in Proc. IEEE 8th Int. Conf. Softw. Testing, Verification Validation Workshops, 2015, pp. 1–10.

[58]

E. Alegroth, Z. Gao, R. Oliveira, and A. Memon, “Conceptualization and evaluation of component-based testing unified with visual GUI testing: An empirical study,” in Proc. IEEE 8th Int. Conf. Softw. Testing, Verification Validation, 2015, pp. 1–10.

[59]

V. Dallmeier, M. Burger, T. Orth, and A. Zeller, “Webmate: A tool for testing web 2.0 applications,” in Proc. Workshop JavaScript Tools, 2012, pp. 11–15.

[60]

A. Marchetto, P. Tonella, and F. Ricca, “State-based testing of ajax web applications,” in Proc. 1st Int. Conf. Softw. Testing, Verification, Validation, 2008, pp. 121–130.

[61]

M. Biagiola, F. Ricca, and P. Tonella, “Search based path and input data generation for web application testing,” in Proc. Int. Symp. Search Based Softw. Eng., 2017, pp. 18–32.

[62]

U. Praphamontripong, J. Offutt, L. Deng, and J. Gu, “An experimental evaluation of web mutation operators,” in Proc. IEEE 9th Int. Conf. Softw. Testing, Verification Validation Workshops, 2016, pp. 102–111.

[63]

Y. Sun, P. Jin, and L. Yue, “A framework of a hybrid focused web crawler,” in Proc. 2nd Int. Conf. Future Gener. Commun. Netw. Symposia, 2008, pp. 50–53.

[64]

A. Milani Fard and A. Mesbah, “Feedback-directed exploration of web applications to derive test models,” in Proc. Int. Symp. Softw. Rel. Eng., 2013, pp. 278–287.

[65]

Y. Zhenget al., “Automatic web testing using curiosity-driven reinforcement learning,” in Proc. IEEE/ACM 43rd Int. Conf. Softw. Eng., 2021, pp. 423–435.

[66]

C. Degott, N. P. Borges Jr., and A. Zeller, “Learning user interface element interactions,” in Proc. 28th ACM SIGSOFT Int. Symp. Softw. Testing Anal., 2019, pp. 296–306.

Cited By

Alian PNashid NShahbandeh MMesbah AChristakis MPradel M(2024)Semantic Constraint Inference for Web Form Test GenerationProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680332(932-944)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680332
Clerissi DDenaro GMobilio MMariani L(2024)Guess the State: Exploiting Determinism to Improve GUI Exploration EfficiencyIEEE Transactions on Software Engineering10.1109/TSE.2024.336658650:4(836-853)Online publication date: 16-Feb-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3366586
Huynh MLe QNguyen VNguyen T(2023)Web Page Segmentation: A DOM-Structural Cohesion Analysis ApproachWeb Information Systems Engineering – WISE 202310.1007/978-981-99-7254-8_25(319-333)Online publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1007/978-981-99-7254-8_25

Recommendations

Diversity-based web test generation
ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Existing web test generators derive test paths from a navigational model of the web application, completed with either manually or randomly generated input values. However, manual test data selection is costly, while random generation often results in ...
Achieving scalable mutation-based generation of whole test suites

Without complete formal specification, automatically generated software tests need to be manually checked in order to detect faults. This makes it desirable to produce the strongest possible test set while keeping the number of tests as small as ...
Model-Based Test Case Generation for Web Applications
Proceedings of the 14th International Conference on Computational Science and Its Applications ICCSA 2014 - Volume 8584

This paper presents a tool to filter/configure the test cases generated within the Model-Based Testing project PBGT. The models are written in a Domain Specific Language called PARADIGM and are composed by User Interface Test Patterns (UITP) describing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 49, Issue 3

March 2023

450 pages

ISSN:0098-5589

Issue’s Table of Contents

0098-5589 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 March 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alian PNashid NShahbandeh MMesbah AChristakis MPradel M(2024)Semantic Constraint Inference for Web Form Test GenerationProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680332(932-944)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680332
Clerissi DDenaro GMobilio MMariani L(2024)Guess the State: Exploiting Determinism to Improve GUI Exploration EfficiencyIEEE Transactions on Software Engineering10.1109/TSE.2024.336658650:4(836-853)Online publication date: 16-Feb-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3366586
Huynh MLe QNguyen VNguyen T(2023)Web Page Segmentation: A DOM-Structural Cohesion Analysis ApproachWeb Information Systems Engineering – WISE 202310.1007/978-981-99-7254-8_25(319-333)Online publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1007/978-981-99-7254-8_25

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents