Guide · How to fix
Publish a robots.txt file
robots.txt tells search engines and AI what they can crawl. Missing, they move blind (or can't find your sitemap).
Step by step
- 1
Create a file reachable at https://your-site.com/robots.txt.
- 2
Healthy minimal content: “User-agent: * / Allow: /” then “Sitemap: https://your-site.com/sitemap.xml”.
- 3
On Shopify/Webflow it exists by default; on WordPress an SEO plugin generates it. If you code, drop it at the domain root.
- 4
Check it doesn't accidentally block important pages (overly broad Disallow).
Generalist method: adapt “depending on your tool” (WordPress, Shopify, Webflow, or directly in code).
Rather we handle it?
We fix this action for you, ready to ship — €37 per action.