Back to Free Tools

Robots.txt Checker


What it is

Robots.txt is a text file that provides instructions to Search Engine crawlers on how to crawl your site, including types of pages to access or not access. It is often the gatekeeper of your site, and normally the first thing a Search Engine bot will access.

How to fix it

We recommend always having a robots file in place for your site. These can be automatically created using a free online utility, Wordpress plugin, or your CMS's robots.txt creation process.

SEOptimer Report Preview

SEOptimer - SEO Audit & Reporting Tool.
Improve Your Website. Win More Customers.

Get a Free Website Audit Instantly