research-article

WATER: Web Application TEst Repair

Authors:

Shauvik Roy Choudhary,

Dan Zhao,

Husayn Versee,

Alessandro OrsoAuthors Info & Claims

ETSE '11: Proceedings of the First International Workshop on End-to-End Test Script Engineering

Pages 24 - 29

https://doi.org/10.1145/2002931.2002935

Published: 17 July 2011 Publication History

Get Access

Abstract

Web applications tend to evolve quickly, resulting in errors and failures in test automation scripts that exercise them. Repairing such scripts to work on the updated application is essential for maintaining the quality of the test suite. Updating such scripts manually is a time consuming task, which is often difficult and is prone to errors if not performed carefully. In this paper, we propose a technique to automatically suggest repairs for such web application test scripts. Our technique is based on differential testing and compares the behavior of the test case on two successive versions of the web application: first version in which the test script runs successfully and the second version in which the script results in an error or failure. By analyzing the difference between these two executions, our technique suggests repairs that can be applied to repair the scripts. To evaluate our technique, we implemented it in a tool called WATER and exercised it on real web applications with test cases. Our experiments show that WATER can suggest meaningful repairs for practical test cases, many of which correspond to those made later by developers themselves.

References

[1]

N. Alshahwan and M. Harman. Automated session data repair for web application regression testing. In Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, pages 298--307, Washington, DC, USA, 2008. IEEE Computer Society.

Digital Library

Google Scholar

[2]

Apache Foundation. The apache open for business project. http://ofbiz.apache.org/, Apr 2011.

Google Scholar

[3]

B. Daniel, T. Gvero, and D. Marinov. On test repair using symbolic execution. In Proceedings of the 19th international symposium on Software testing and analysis, ISSTA '10, pages 207--218, New York, NY, USA, 2010. ACM.

Digital Library

Google Scholar

[4]

B. Daniel, V. Jagannath, D. Dig, and D. Marinov. Reassert: Suggesting repairs for broken unit tests. In Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering, ASE '09, pages 433--444, Washington, DC, USA, 2009. IEEE Computer Society.

Digital Library

Google Scholar

[5]

K. Dobolyi and W. Weimer. Harnessing web-based application similarities to aid in regression testing. In Proceedings of the 20th IEEE international conference on software reliability engineering, ISSRE'09, pages 71--80, Piscataway, NJ, USA, 2009. IEEE Press.

Digital Library

Google Scholar

[6]

M. Grechanik, Q. Xie, and C. Fu. Maintaining and evolving gui-directed test scripts. In Proceedings of the 31st International Conference on Software Engineering, ICSE '09, pages 408--418, Washington, DC, USA, 2009. IEEE Computer Society.

Digital Library

Google Scholar

[7]

IBM Research. Coscripter. http://coscripter.researchlabs.ibm.com/coscripter/, Apr 2011.

Google Scholar

[8]

G. Leshed, E. M. Haber, T. Matthews, and T. Lau. Coscripter: automating & sharing how-to knowledge in the enterprise. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, CHI '08, pages 1719--1728, New York, NY, USA, 2008. ACM.

Digital Library

Google Scholar

[9]

A. M. Memon and M. L. Soffa. Regression testing of guis. In Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering, ESEC/FSE-11, pages 118--127, New York, NY, USA, 2003. ACM.

Digital Library

Google Scholar

[10]

Open Source Matters, Inc. Joomla! http://joomla.org/, Apr 2011.

Google Scholar

[11]

OpenQA. Selenium web application testing system. http://seleniumhq.org/, May 2010.

Google Scholar

[12]

D. Roest, A. Mesbah, and A. v. Deursen. Regression testing ajax applications: Coping with dynamism. In Software Testing, Verification and Validation (ICST), 2010 Third International Conference on, pages 127--136, 6--10 2010.

Digital Library

Google Scholar

Cited By

View all

Di Meglio SStarace L(2024)Towards Predicting Fragility in End-to-End Web TestsProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661179(387-392)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661179
Cao SPan MPei YYang WZhang TWang LLi XRoychoudhury APaiva AAbreu RStorey M(2024)Comprehensive Semantic Repair of Obsolete GUI Test Scripts for Mobile ApplicationsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639108(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639108
De Luca MFasolino ATramontana P(2024)Investigating the robustness of locators in template-based Web application testing using a GUI change classification modelJournal of Systems and Software10.1016/j.jss.2023.111932210:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.jss.2023.111932
Show More Cited By

Index Terms

WATER: Web Application TEst Repair
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Visual web test repair
ESEC/FSE 2018: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Web tests are prone to break frequently as the application under test evolves, causing much maintenance effort in practice. To detect the root causes of a test breakage, developers typically inspect the test's interactions with the application through ...
Vista: web test repair using computer vision
ESEC/FSE 2018: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Repairing broken web element locators represents the major main- tenance cost of web test cases. To detect possible repairs, testers typically inspect the tests’ interactions with the application under test through the GUI. Existing automated test ...
Semantic Test Repair for Web Applications
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Automation testing is widely used in the functional testing of web applications. However, during the evolution of web applications, such web test scripts tend to break. It is essential to repair such broken test scripts to make regression testing run ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ETSE '11: Proceedings of the First International Workshop on End-to-End Test Script Engineering

July 2011

46 pages

ISBN:9781450308083

DOI:10.1145/2002931

Conference Chairs:
Mark Grechanik
Accenture Technology Labs
,
Satish Chandra
IBM T.J. Watson Research Center
,
Saurabh Sinha
IBM Research - India

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 July 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '11

Sponsor:

SIGSOFT

ISSTA '11: International Symposium on Software Testing and Analysis

July 17, 2011

Ontario, Toronto, Canada

Upcoming Conference

ISSTA '24

Sponsor:
sigsoft

33rd ACM SIGSOFT International Symposium on Software Testing and Analysis

September 16 - 20, 2024

Vienna , Austria

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

71
Total Citations
View Citations
611
Total Downloads

Downloads (Last 12 months)54
Downloads (Last 6 weeks)1

Reflects downloads up to 14 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Di Meglio SStarace L(2024)Towards Predicting Fragility in End-to-End Web TestsProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661179(387-392)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661179
Cao SPan MPei YYang WZhang TWang LLi XRoychoudhury APaiva AAbreu RStorey M(2024)Comprehensive Semantic Repair of Obsolete GUI Test Scripts for Mobile ApplicationsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639108(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639108
De Luca MFasolino ATramontana P(2024)Investigating the robustness of locators in template-based Web application testing using a GUI change classification modelJournal of Systems and Software10.1016/j.jss.2023.111932210:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.jss.2023.111932
Khalili FMariani LMohebbi APezzè MTerragni V(2024)Semantic matching in GUI test reuseEmpirical Software Engineering10.1007/s10664-023-10406-829:3Online publication date: 9-May-2024
https://doi.org/10.1007/s10664-023-10406-8
Jiang PWang SLiu Y(2024)Tree-Based Synthesis of Web Test Sequences from Manual ActionsTheoretical Aspects of Software Engineering10.1007/978-3-031-64626-3_14(242-260)Online publication date: 14-Jul-2024
https://doi.org/10.1007/978-3-031-64626-3_14
Nass MAlégroth EFeldt R(2024)Improving Web Element Localization by Using a Large Language ModelSoftware Testing, Verification and Reliability10.1002/stvr.1893Online publication date: 15-Aug-2024
https://doi.org/10.1002/stvr.1893
Zhu HWei LTerragni VLiu YCheung SWu JSheng QZhang BSong L(2023) StubCoder: Automated Generation and Repair of Stub Code for Mock ObjectsACM Transactions on Software Engineering and Methodology10.1145/361717133:1(1-31)Online publication date: 21-Aug-2023
https://dl.acm.org/doi/10.1145/3617171
Qi XQian XLi YChandra SBlincoe KTonella P(2023)Semantic Test Repair for Web ApplicationsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616324(1190-1202)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616324
Nass MAlégroth EFeldt RCoppola R(2023)Robust web element identification for evolving applications by considering visual overlaps2023 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST57152.2023.00032(258-268)Online publication date: Apr-2023
https://doi.org/10.1109/ICST57152.2023.00032
Lin YWen GGao X(2023)Automated Fixing of Web UI Tests via Iterative Element Matching2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00048(1188-1199)Online publication date: 11-Sep-2023
https://doi.org/10.1109/ASE56229.2023.00048
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Visual web test repair

Vista: web test repair using computer vision

Semantic Test Repair for Web Applications