---
title: "#oai-searchbot — seo-mcp"
description: "SEO tips tagged OAI-SearchBot."
url: "https://seo-mcp.io/tag/oai-searchbot"
---

# #oai-searchbot

2 published tips tagged “OAI-SearchBot”.

- [Allow AI crawlers in robots](https://seo-mcp.io/t/allow-ai-crawlers-in-robots) — AI assistants fetch pages with their own user agents: GPTBot and OAI-SearchBot from OpenAI, ClaudeBot from Anthropic, PerplexityBot, and Google-Extended for Gemini. Decide per bot between training and retrieval, write explicit groups, and test with curl using each agent string.
- [Allow AI crawlers in robots.txt so ChatGPT can cite the site](https://seo-mcp.io/t/allow-ai-crawlers-in-robots-txt-so-chatgpt-can-cite-the-site) — Review robots.txt for Disallow rules aimed at AI crawler user agents, which many sites added in 2024, and replace them with explicit Allow rules for the pages you want cited. Then check that your CDN or WAF does not block the same bots at the edge, since a robots.txt fix alone will not help if the request never reaches the origin.