Crawlability check

Robots.txt and Sitemap Validator

Check whether search engines can crawl your site, read your robots.txt rules, and discover valid sitemap URLs.

SEO Robots + Sitemap Find crawl blocking issues before search engines do.
User-agent: *Provide a default block for general crawlers.
List sitemapAdd your sitemap URL inside robots.txt.
Avoid Disallow: /Use it only when you intentionally want to block all crawling.