Describe the bug

UserAgentScanRule is a bit overzealous. If it was used against an endpoint /clock which returned the current time or /random that returned a random number, or a page that returned a random /phrase, it'd complain that different user-agents were getting different content.

Similarly, if it was run against /what-is-my-user-agent/ it would complain that different user-agents receive different content -- this is technically true, but not meaningfully true.

Steps to reproduce the behavior

1. Pick an endpoint that returns different content based on time of day (or just returns an internal request id, or a random number/string)
2. Trigger the UserAgentScanRule
3. Look at the report

Expected behavior

1. UserAgentScanRule should make a couple of requests to gather a baseline:
   • request at time t=0 with user-agent hello-world/0.1
   • request at time t=1 with user-agent hello-world/0.1 just to check to see if clock ticks are enough to trigger changes
   • request at time t=2 with user-agent world-hello/0.1 (to see if the useragent is being reported without having any particularly meaningful impact on the page)
2. If the responses are all the same, it can compare any varied user-agent request against the first response and complain if things differ.
3. If the responses are not the same, it should only compare varied user-agent responses against the parts that are the same between the baseline requests

Given that the test makes ~12 calls per base url including distinct calls for http://example.com and http://example.com/, making 2 extra calls to establish a useful baseline shouldn't be a particularly big deal.

Software versions

ZAP
Version: 2.15.0

Installed Add-ons: [[id=accessControl, version=10.0.0],
[id=alertFilters, version=21.0.0], [id=ascanrules,
version=67.0.0], [id=ascanrulesBeta, version=54.0.0],
[id=authhelper, version=0.14.0], [id=authstats,
version=2.0.0], [id=automation, version=0.41.0],
[id=bruteforce, version=16.0.0], [id=callhome,
version=0.12.0], [id=commonlib, version=1.26.0],
[id=custompayloads, version=0.13.0], [id=database,
version=0.5.0], [id=diff, version=15.0.0],
[id=directorylistv1, version=8.0.0], [id=directorylistv2_3,
version=4.0.0], [id=domxss, version=19.0.0], [id=encoder,
version=1.5.0], [id=exim, version=0.10.0], [id=fileupload,
version=1.2.1], [id=formhandler, version=6.6.0], [id=fuzz,
version=13.13.0], [id=fuzzdb, version=9.0.0],
[id=fuzzdboffensive, version=5.0.0], [id=gettingStarted,
version=17.0.0], [id=graaljs, version=0.7.0], [id=graphql,
version=0.24.0], [id=help, version=18.0.0], [id=hud,
version=0.19.0], [id=importurls, version=9.0.0], [id=invoke,
version=15.0.0], [id=jsonview, version=3.0.0], [id=jwt,
version=1.0.3], [id=network, version=0.16.0], [id=oast,
version=0.19.0], [id=onlineMenu, version=13.0.0],
[id=openapi, version=42.0.0], [id=postman, version=0.4.0],
[id=pscanrules, version=59.0.0], [id=quickstart,
version=48.0.0], [id=replacer, version=18.0.0], [id=reports,
version=0.32.0], [id=requester, version=7.7.0], [id=retest,
version=0.9.0], [id=retire, version=0.38.0], [id=reveal,
version=8.0.0], [id=saml, version=10.0.0],
[id=saverawmessage, version=7.0.0], [id=savexmlmessage,
version=0.3.0], [id=scripts, version=45.5.0], [id=selenium,
version=15.27.0], [id=soap, version=23.0.0], [id=spider,
version=0.11.0], [id=spiderAjax, version=23.20.0],
[id=sqliplugin, version=15.0.0], [id=sse, version=13.0.0],
[id=tips, version=13.0.0], [id=wappalyzer, version=21.39.0],
[id=webdrivermacos, version=98.0.0], [id=webdriverwindows,
version=98.0.0], [id=websocket, version=31.0.0], [id=zest,
version=46.0.0]]

Operating System: Mac OS X
Architecture: aarch64
CPU Cores: 10
Max Memory: 16 GB
Java Version: Eclipse Adoptium 11.0.23
System's Locale: en_US
Display Locale: en_GB
Format Locale: en_US
Default Charset: UTF-8
ZAP Home Directory: /Users/jsoref/Library/Application Support/ZAP/
ZAP Installation Directory: /Applications/ZAP.app/Contents/Java/./
Look and Feel: FlatLaf Light (com.formdev.flatlaf.FlatLightLaf)

Screenshots

Errors from the zap.log file

No response

Additional context

https://github.com/zaproxy/zap-extensions/blob/b24a40efae10a260b8225a3f8e1a62ac4ca87170/addOns/ascanrules/src/main/java/org/zaproxy/zap/extension/ascanrules/UserAgentScanRule.java#L145

• I worked on a tool like this one in 2004/2005 and we had this functionality to avoid the false positives reported by this checker.
• I was working on a major web browser at the time and we were certainly very sensitive about discrimination by websites, the things I'm describing would not count as discrimination.

Would you like to help fix this issue?

• Yes