What is a robots.txt file?

Robots.txt is a file that tells search engine crawlers which pages or sections of your site they can or cannot crawl and index. It helps manage crawl budget and prevent indexing of private pages.

Loading...

Back to Tools

Robots.txt Generator

Create a robots.txt file for your website.

User-agent:

Disallow:

Allow:

Crawl-delay (seconds)

Sitemap URL

Generated robots.txt

# robots.txt generated by Kazi Mamun Tools
User-agent: *

Share:

🔒

100% Secure

Files are processed in your browser. Nothing is uploaded.

⚡

Lightning Fast

Process files instantly with no waiting time.

💯

Always Free

All tools are completely free with no limitations.