robots.txt Generator

Build robots.txt files with user-agent rules and sitemap references.

Sitemap URL

Crawl Delay (seconds)

Rule #1

User-agent

Allow (one per line)/

Disallow (one per line)/admin/ /private/

+ Add Rule

robots.txt OutputCopy

User-agent: * Allow: / Disallow: /admin/ Disallow: /private/ Sitemap: https://example.com/sitemap.xml