WebExplor - Experiments

How effective WebExplor is in detecting web bugs?

Research Questions

RQ1 (Code Coverage) How is the exploration capability of WebExplor in terms of code coverage?
RQ2 (Failure Detection) How effective is WebExplor for detecting failures of web applications?
RQ3 (DFA Guidance) How effective is DFA in guiding the exploration during testing?
RQ4 (Scalability) How effective is WebExplor in testing real-world web applications?

Baselines

[Model-Based] DIG leverages a diversity-based test case generator for web testing.
[Model-Based] SUBWEB considers the uncovered branches and using a search-based strategy to achieve web testing.
[Model-Free] Crawljax discovers and clusterspages on the ﬂy and adopts a crawlingbased random test case generator for web testing.
[Model-Free] Random randomly selects one of the available actions to explore web states.
[Model-Free] WebExplor (no DFA) is a variant of WebExplor without the DFA guidance.

Code Coverage / Failure Detection on Benchmark (RQ 1&2)

In contrast to model-based approaches, WebExplor achieves better code coverage and robustness in most cases with no navigation models or prior knowledge.
WebExplor achieves the best performance in failure detection, while requiring no human knowledge or ﬁne-tuned navigation models, revealing its potentials across different subjects.
WebExplor performs good scalability on all 6 benchmark projects.

Failures in Splittypie

Overall, WebExplor discovered more failures (in bold) than failures than other baselinse. Note that, newly discovered failures are mainly server request errors (e.g., CROS or AbortError), which are spotted via illegal operation sequences.

Failures list (by WebExplor):

[1] SEVERE (New by WebExplor)------>http://localhost:4200/assets/splittypie.js 1905:63 TypeError: Cannot read property 'focus' of undefined[2] SEVERE (New by WebExplor)------>http://localhost:4200/assets/splittypie.js 3400:54 AbortError[3] SEVERE (New by WebExplor)------>http://localhost:49153/livereload.js - Failed to load resource: net::ERR_CONNECTION_REFUSED[4] SEVERE (New by WebExplor)------>http://localhost:4200/assets/splittypie.js 1905:106 "TypeError: Cannot read property 'focus' of undefined at http://localhost:4200/assets/splittypie.js:307:62"[5] SEVERE (New by WebExplor)------>http://localhost:4200/-M6XnSPv1KFZOGa1bvGs - Access to XMLHttpRequest at 'https://geoip.nekudo.com/api' from origin 'http://localhost:4200' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.[6] SEVERE (New by WebExplor)------>http://localhost:4200/ - Access to XMLHttpRequest at 'https://geoip.nekudo.com/api' from origin 'http://localhost:4200' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
[7] SEVERE------>http://localhost:4200/assets/splittypie.js 1905:63 Error: FIREBASE FATAL ERROR: Cannot parse Firebase url. [8] SEVERE------>http://localhost:4200/assets/splittypie.js 1905:106 "Error: FIREBASE FATAL ERROR: Cannot parse Firebase url. Please use https://\u003CYOUR FIREBASE>.firebaseio.com[9] SEVERE------>http://localhost:4200/assets/splittypie.js 1905:63 EmberError[10]SEVERE------>http://localhost:4200/assets/splittypie.js 1905:106 "Error: Nothing handled the action 'delete'. If you did handle the action, this error can be caused by returning true from an action handler in a controller, causing the action to bubble."

Conclusion: Intuitively, the code related to each operation can be easily covered by independent execution. However, some failures are only triggered by illegal operation orders, indicating that it seems ineffective to detect failures by simply improving code coverage.

Efficiency of DFA (RQ 3)

In WebExplor, DFA complements curiosity-driven RL by providing high-level guidance for effective exploration and high efﬁciency.
WebExplor achieves a better performance regarding failure detection rate, code coverage and stability than the one without DFA.

Page updated

Google Sites

Report abuse