Compare crawlers
GPTBot vs OAI-SearchBot.
Two OpenAI crawlers with two different jobs, and two separate robots.txt tokens.
What each one does, per OpenAI
GPTBot
OpenAI says GPTBot is used to make its generative AI foundation models more useful and safe, and that it crawls content that may be used to train those models. Disallowing it indicates a site's content should not be used in training.
OAI-SearchBot
OpenAI says OAI-SearchBot is for search, used to surface websites in ChatGPT's search features. Sites that opt out will not appear in ChatGPT search answers, though they can still appear as navigational links. OpenAI recommends allowing it.
Neither one governs the other. Blocking GPTBot does not remove you from ChatGPT search, and blocking OAI-SearchBot does not opt you out of training.
Allow one, block the other
OpenAI's page names the two tokens but prints no example lines, so these are standard robots.txt syntax around those tokens.
Appear in ChatGPT search, stay out of training
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /Allow training, stay out of ChatGPT search
User-agent: GPTBot
Allow: /
User-agent: OAI-SearchBot
Disallow: /OpenAI says search-related robots.txt changes can take about 24 hours to take effect. A third OpenAI agent, ChatGPT-User, handles actions a person takes in ChatGPT, and OpenAI says robots.txt rules may not apply to it.
What Geordy observes
Not measured yet. Geordy will publish what these crawlers request from Geordy-served files once enough independent sites are covered. See how we measure.
Check your site
See which of these crawlers your robots.txt allows today.
Sources
Last reviewed: 19 October 2026. Every statement about what each crawler does comes from OpenAI's documentation: