How to Generate a robots.txt File Online for Free

Step 1: Configure Allow and Disallow Rules

Set which bots to target (use * for all), then add the URL paths you want to allow or disallow crawling for.

Step 2: Add Your Sitemap URL

Enter the full URL of your sitemap.xml file so search engines can discover it from your robots.txt automatically.

Step 3: Preview the Generated File

The robots.txt content updates live as you configure rules, showing you exactly what crawlers will see.

Step 4: Download the File

Click Download to save the robots.txt file and upload it to the root of your website domain.

Your URL structure and crawl rules never leave your browser — no third party sees your site configuration.

The robots.txt output updates in real time as you add rules so you know exactly what you're deploying.

Download the robots.txt file directly and upload it to your website root.