Retrieval & Behavior· Topic 07 · Definition

User-Agent

The line a browser or crawler sends with every request, naming itself. It is how a visit from GPTBot is told apart from a visit from a person.

Every HTTP request carries a User-Agent header, a short self-declared string such as Mozilla/5.0 (compatible; GPTBot/1.2; +https://openai.com/gptbot). Bot analytics is built on it: match the string against a list of known crawlers and you know which AI system read which file and when. It is self-declared, so it can be faked, but the major AI operators publish their agent names precisely so publishers can recognise and control them. The same names appear on the User-agent lines in robots.txt.

Related terms
GEO StrategyAI IndexabilityWhether AI engines can actually parse and store your content.Emerging ConceptsAgentic DiscoveryWhen autonomous AI agents browse and choose your site without a human in the loop.Technical ConstraintsAI Crawl Budget (ACB)How much an AI engine is willing to fetch from your site within a window.
More in Retrieval & BehaviorAll 7 terms →
Content Chunking Context Window Fit Embedding Relevance Freshness Signals (AI) Query Intent Matching (LLM-Level) Tool-Augmented Retrieval

Knowing the term is step one.

Geordy operationalizes every term in this glossary - generating the structured files AI engines actually read.

Start free →Back to glossary
User-Agent | Geordy Glossary