GPTBot vs OAI-SearchBot comes down to one question: training or search? GPTBot collects pages that may be used to train OpenAI's future models. OAI-SearchBot finds pages so ChatGPT search can show and link them. You can block GPTBot and still appear in ChatGPT. Block OAI-SearchBot and OpenAI says your site "will not be shown in ChatGPT search answers."
OpenAI actually runs four user agents, not two. Each one has a different job and needs a different decision. This post goes through all four, says which one you can safely block and which one you should never block, and gives you robots.txt rules to copy.
What are all of OpenAI's crawlers?
As of 3 October 2026, OpenAI's crawler documentation lists four user agents. OpenAI describes them as robots and user agents that "perform actions for its products, either automatically or triggered by user request."
| User agent | What it does | Used for training? | Follows robots.txt? | Our advice for D2C brands |
|---|---|---|---|---|
OAI-SearchBot |
Finds pages to show and link in ChatGPT search | Not its stated job (it's described as "for search") | Yes | Never block |
GPTBot |
Crawls content that may be used to train OpenAI's foundation models | Yes | Yes | Your call. Most brands should allow it |
ChatGPT-User |
Visits a page when someone in ChatGPT or a Custom GPT asks a question or takes an action | Not mentioned | "Robots.txt rules may not apply" | Don't block |
OAI-AdsBot |
Checks landing pages submitted as ChatGPT ads | No, OpenAI says its data isn't used for training | Not stated | Leave alone; it only visits your ad pages |
The rest of this post goes through them one by one.
What does OAI-SearchBot do?
OAI-SearchBot is the bot that gets you into ChatGPT's answers. OpenAI's description: it "is used to surface websites in search results in ChatGPT's search features."
When a buyer asks ChatGPT "best vitamin C serum under ₹800 for oily skin" and ChatGPT searches the web, the pages it can cite come from what OAI-SearchBot has indexed. If you block it, OpenAI is blunt about the result: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers."
That's why this is the one OpenAI bot you should never block. It isn't a training crawler. It's closer to Googlebot for ChatGPT. Blocking it is like asking Google to drop you from search because you don't like AI.
Its full user agent string currently looks like this:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Notice it starts like a normal Chrome browser. Firewalls that only look at the start of the string can miss it, and rules that block "headless Chrome" can catch it by accident.
What does GPTBot do?
GPTBot is OpenAI's training crawler. OpenAI says it "is used to crawl content that may be used in training our generative AI foundation models." Blocking it, in OpenAI's words, "indicates a site's content should not be used in training generative AI foundation models."
Two things to understand before you decide:
- Blocking GPTBot does not remove you from ChatGPT search. That's OAI-SearchBot's job, and the two are controlled separately.
- Blocking GPTBot is about the future, not the past. It tells OpenAI not to use your content in training from now on. It doesn't pull your pages out of models that were already trained.
For most D2C brands, we think allowing GPTBot makes sense. Your product pages, ingredient lists and FAQs are marketing. You want future models to know that your brand makes a high-protein bar with no added sugar, or that your sunscreen is SPF 50 PA++++. Being in training data is no guarantee of that, but blocking makes it less likely.
There are fair reasons to block it. Maybe your content is the thing you sell: a paid recipe library, a course, original research. Or your legal team has a policy on training data. In those cases, block GPTBot and keep OAI-SearchBot allowed. That's the whole point of OpenAI splitting them.
What does ChatGPT-User do?
ChatGPT-User isn't a crawler in the usual sense. OpenAI says it is used "for certain user actions in ChatGPT and Custom GPTs. When users ask ChatGPT or a CustomGPT a question, it may visit a web page with a ChatGPT-User agent." It also covers GPT Actions, where ChatGPT calls external apps.
Two lines in the docs matter here:
- "Because these actions are initiated by a user, robots.txt rules may not apply."
- "ChatGPT-User is not used to determine whether content may appear in Search."
So a robots.txt block on ChatGPT-User may not be honoured. Even if it is, it won't change whether you appear in ChatGPT search. What it can do is stop ChatGPT opening your page when a real person asks it to. That person is often a shopper comparing you with a competitor. There's very little reason to block that.
What is OAI-AdsBot?
OAI-AdsBot is the newest one on the list. OpenAI says it "only visits pages submitted as ads," and "the data collected by OAI-AdsBot is not used to train generative AI foundation models." If you never run ads in ChatGPT, you'll probably never see it.
If you do run ChatGPT ads, make sure your landing pages don't block it, whether through robots.txt, a firewall rule or a bot-protection app. A validation bot that can't load your page is a good way to get an ad rejected. The docs don't say whether it reads robots.txt, so the safe move is simply not to block it.
GPTBot vs OAI-SearchBot: which should you block?
Here's the decision for a typical D2C brand:
| Your situation | GPTBot | OAI-SearchBot | ChatGPT-User | OAI-AdsBot |
|---|---|---|---|---|
| You want to be found and known by AI (most brands) | Allow | Allow | Allow | Allow |
| You want ChatGPT citations but not training | Block | Allow | Allow | Allow |
| One section is premium content (recipes, courses) | Block that folder | Allow | Allow | Allow |
| You want nothing to do with OpenAI at all | Block | Block (you leave ChatGPT search) | Can't fully control | Not relevant |
The last row is rarely the right call for a brand that sells things. You'd be removing yourself from a channel your buyers already use and getting nothing back.
One useful detail from OpenAI's docs: "If your site has allowed both bots, we may use the results from just one crawl for both use cases to avoid duplicative crawling." So allowing both doesn't necessarily mean twice the server load.
Copy-paste robots.txt rules for OpenAI's bots
Replace the paths with your own. Shopify, WooCommerce and custom sites use different cart and account URLs.
Allow everything (the default we recommend)
User-agent: *
Disallow: /cart
Disallow: /checkout
Disallow: /account
Sitemap: https://www.yourbrand.com/sitemap.xml
You don't need to name OpenAI's bots to allow them. Any bot without its own group follows User-agent: *.
Block GPTBot, keep ChatGPT search
User-agent: GPTBot
Disallow: /
User-agent: *
Disallow: /cart
Disallow: /checkout
Disallow: /account
Sitemap: https://www.yourbrand.com/sitemap.xml
OAI-SearchBot isn't named, so it follows the * group and can crawl your store.
Block GPTBot from one folder only
User-agent: GPTBot
Disallow: /recipes/
Disallow: /cart
Disallow: /checkout
Disallow: /account
User-agent: *
Disallow: /cart
Disallow: /checkout
Disallow: /account
The cart rules are repeated on purpose. Google's robots.txt documentation explains why: "Only one group is valid for a particular crawler." A bot with its own group ignores the * group completely. If you leave the cart rules out of the GPTBot group, GPTBot is allowed into your cart pages.
The mistake to avoid
User-agent: GPTBot
User-agent: OAI-SearchBot
Disallow: /
This is what happens when someone copies a "block OpenAI" snippet and adds OAI-SearchBot "to be thorough". It blocks training and also removes the site from ChatGPT search. If your file has OAI-SearchBot next to Disallow: /, that's almost certainly not what you meant.
Anthropic, Perplexity, Google and Apple each split their bots in a similar way, so check those groups too if you've named them.
How do you know a visit is really from OpenAI?
Anyone can put "GPTBot" in a user agent string. Scrapers sometimes copy it so they get waved through. OpenAI publishes the IP ranges for each bot, so you or your developer can check:
- GPTBot:
https://openai.com/gptbot.json - OAI-SearchBot:
https://openai.com/searchbot.json - ChatGPT-User:
https://openai.com/chatgpt-user.json - OAI-AdsBot:
https://openai.com/adsbot.json
If a request says it's GPTBot but comes from an IP outside that list, it isn't GPTBot. Block it at the firewall if you like. Your robots.txt decision about the real one stays the same.
Most CDNs and bot-protection tools, such as Cloudflare, can verify known bots for you. Check that their "AI bot" settings match your robots.txt. A common failure is a one-click "block AI crawlers" toggle that also blocks OAI-SearchBot. Your robots.txt says allow, but the bot gets an error page anyway.
How to check yours
This takes about ten minutes.
- Open
https://yourbrand.com/robots.txtand search forGPTBot,OAI-SearchBot,ChatGPT-UserandOAI-AdsBot. - If
OAI-SearchBotappears in a group withDisallow: /, remove it. That one line keeps you out of ChatGPT search. - If
GPTBothas its own group, check that the group also repeats your cart, checkout and account rules. - Look for
User-agent: *withDisallow: /. That blocks every bot that isn't named, including OAI-SearchBot. - Run your domain through our free AI crawler checker. It reads your live robots.txt and shows which AI search and training bots, including GPTBot and OAI-SearchBot, can reach your pages.
- Check your CDN, firewall and any Shopify or WordPress security app for AI-bot blocking.
- Give changes time. OpenAI says "it can take ~24 hours from a site's robots.txt update for our systems to adjust."
Crawler access is the first step of AI search visibility. Once OAI-SearchBot can get in, the next question is whether ChatGPT actually picks you. Our post on why your brand doesn't show up on ChatGPT covers that.
FAQ
Is OAI-SearchBot the same as GPTBot?
No. OAI-SearchBot finds pages for ChatGPT search results. GPTBot crawls content that may be used to train OpenAI's foundation models. They have separate user agents, and you control each one separately in robots.txt.
If I block GPTBot, will my brand disappear from ChatGPT?
Not from ChatGPT search. Search uses OAI-SearchBot, so as long as that's allowed, your pages can still be cited and linked. Blocking GPTBot only signals that your content shouldn't be used to train future models.
Can I block ChatGPT-User?
You can add a rule, but OpenAI says that because these visits are triggered by users, "robots.txt rules may not apply." ChatGPT-User also isn't used to decide whether you appear in search. Blocking it mostly stops ChatGPT opening your page for a shopper who asked about you.
What is OAI-AdsBot and should I worry about it?
It checks landing pages submitted as ChatGPT ads, and OpenAI says its data isn't used for training. It only visits ad pages, so if you don't run ChatGPT ads it won't affect you. If you do, make sure nothing blocks it.
Does allowing both GPTBot and OAI-SearchBot double my server load?
Not necessarily. OpenAI says that if you allow both, it may use one crawl for both purposes to avoid crawling twice.
If you'd like to see what happens once the bots are in, including which buyer questions in ChatGPT mention your brand and which mention competitors, get the free AI visibility report. Or talk to us if you'd like someone to look over your setup.