Skip to content
Technical

robots.txt Tester

Validate directives and test paths against your robots.txt in the browser.

Paste a robots.txt and a URL path; get back which user-agents can crawl it, which directive matched, and any syntax warnings.

Verdict
allowed
Matched rule: allow: /admin/public/
Useful for
  • ·Debugging crawler access issues
  • ·Validating a robots.txt before deploy
  • ·Confirming Googlebot behaviour
Other tools
Austin & Texas coverage

Local SEO teams we work with.

Need this at scale?

We run this analysis across thousands of pages in engagements.