Visual web test repair

A Stocco, R Yandrapally, A Mesbah - Proceedings of the 2018 26th ACM …, 2018 - dl.acm.org
Proceedings of the 2018 26th ACM Joint Meeting on European Software …, 2018dl.acm.org
Web tests are prone to break frequently as the application under test evolves, causing much
maintenance effort in practice. To detect the root causes of a test breakage, developers
typically inspect the test's interactions with the application through the GUI. Existing
automated test repair techniques focus instead on the code and entirely ignore visual
aspects of the application. We propose a test repair technique that is informed by a visual
analysis of the application. Our approach captures relevant visual information from tests …
Web tests are prone to break frequently as the application under test evolves, causing much maintenance effort in practice. To detect the root causes of a test breakage, developers typically inspect the test's interactions with the application through the GUI. Existing automated test repair techniques focus instead on the code and entirely ignore visual aspects of the application. We propose a test repair technique that is informed by a visual analysis of the application. Our approach captures relevant visual information from tests execution and analyzes them through a fast image processing pipeline to visually validate test cases as they re-executed for regression purposes. Then, it reports the occurrences of breakages and potential fixes to the testers. Our approach is also equipped with a local crawling mechanism to handle non-trivial breakage scenarios such as the ones that require to repair the test's workflow. We implemented our approach in a tool called Vista. Our empirical evaluation on 2,672 test cases spanning 86 releases of four web applications shows that Vista is able to repair, on average, 81% of the breakages, a 41% increment with respect to existing techniques.
ACM Digital Library