Search indexing crawler
OAI-SearchBot: what it is.
OAI-SearchBot is OpenAI's crawler for surfacing websites in ChatGPT's search features.
Identity
- Operator: OpenAI
- User-agent token:
OAI-SearchBot - robots.txt token:
OAI-SearchBot - In Geordy's crawler directory: Indexes pages so ChatGPT search can surface them. See the full list.
What it does, per OpenAI
OpenAI says OAI-SearchBot is for search, and that it is used to surface websites in ChatGPT's search features.
OpenAI says sites that opt out of OAI-SearchBot will not appear in ChatGPT search answers, though they can still appear as navigational links.
OpenAI recommends allowing OAI-SearchBot in robots.txt and allowing requests from the IP ranges it publishes at openai.com/searchbot.json.
robots.txt
Allow OAI-SearchBot
User-agent: OAI-SearchBot
Allow: /Block OAI-SearchBot
User-agent: OAI-SearchBot
Disallow: /OpenAI names OAI-SearchBot as the robots.txt token. Its page does not print example lines, so the ones below are standard robots.txt syntax around that token.
OpenAI says changes to search-related robots.txt rules can take about 24 hours to take effect.
OpenAI says ChatGPT-User does not decide whether content appears in Search, and points to OAI-SearchBot in robots.txt for managing Search opt-outs.
Allowing one and blocking the other is covered in GPTBot vs OAI-SearchBot.
What Geordy observes
Share of eligible sites where OAI-SearchBot requested each Geordy-served file at least once, over the 90 days to 12 November 2026.
| Format | Evidence | Sites with a request | Median requests per site per day | Rank |
|---|---|---|---|---|
| Markdown | Established pattern | under 5% | 0 | Similar to others |
| Schema JSON (JSON-LD) | Established pattern | under 5% | 0 | Similar to others |
| OG.json | Established pattern | under 5% | 0 | Similar to others |
| YAML | Established pattern | about 5% | 0 | Similar to others |
| LLMs.txt | Established pattern | 0% | 0 | Similar to others |
| Humans.txt | Established pattern | 0% | 0 | Similar to others |
| Manifest.json | Established pattern | 0% | 0 | Similar to others |
| RSS Feed | Established pattern | 0% | 0 | Similar to others |
Figures are rounded and cover only sites that served the file for the whole window. Requests are to Geordy-served files, identified by user agent; a request is not evidence of use or citation. Pairs with too few independent sites are not shown. See how we measure.
Check your site
See whether your robots.txt currently allows or blocks OAI-SearchBot, and what else AI crawlers find on your site.
Sources
Last reviewed: 19 October 2026. Every statement about what OAI-SearchBot does comes from the operator's own documentation: