Robots
5 tips · #robots
Allow AI crawlers in robotsShip app/sitemap.ts and app/robots.tshighWrite a crawlable robots.txtAllow AI crawlers in robots.txt so ChatGPT can cite the sitehighAudit Googlebot crawl behavior with server logs to find crawl traps
Let your agent apply every Robots tip.
The MCP pass turns this list into checked-off work in your repo — $19 once.