SI Readiness by MCP-Tool
Agent user agents are actually served
Does the server answer when the request carries an agent user agent?
Why it matters
robots.txt is a request; bot management is enforcement. A WAF rule or rate limiter can return 403 to exactly the clients you meant to welcome, and the permission you wrote is then worth nothing.
How to fix it
Allowlist the agent user agents in your bot management, or verify them by reverse DNS instead of blocking on the string. Then re-test.