How ChatGPT Search Works: How It Finds, Indexes, and Cites Websites
A washing machine manual for one exact model. This year’s passport fees. A restaurant’s current menu and opening hours. Those are the examples Cloudflare and OpenAI used on October 15, 2026 to describe the pages ChatGPT kept missing: useful, specific, and linked to by almost nobody.
Their joint pilot, which used about 1% of Cloudflare’s network, brought “hundreds of millions” of previously unknown pages into OpenAI’s search index. For a small business with a modest website and few backlinks, that is a hint about how ChatGPT search actually works, and about where your pages can fall out of it.
This explainer walks through the whole path, from the moment ChatGPT decides to search to the moment it cites (or skips) your page, using only what OpenAI and Cloudflare have published.
Stage 1: ChatGPT decides whether to search at all
Not every answer comes from the web. According to OpenAI’s help page on searching the web, ChatGPT “may search the web automatically when your question would benefit from current information,” and people can also force a search with the Search tool or by typing /.
Two details on that page matter for local businesses:
- Search is open to almost everyone. Web search is available on every plan OpenAI lists, from Free to Enterprise, and OpenAI says “people who are not signed in can also use web search.”
- Location is part of the query. ChatGPT may estimate a general location from the user’s IP address, and can use a more precise device location if the person turns it on. A question like “good plumber near me” becomes a local search even if the city is never typed.
When ChatGPT answers without searching, it relies on what the model already learned in training. That is the slower-moving side of AI answers, the part that changes only when models are retrained. The rest of this post is about the live side.
Stage 2: The question is rewritten into search queries
ChatGPT doesn’t send your customer’s sentence to a search engine word for word. OpenAI says it “typically rewrites your query into one or more targeted queries,” and may send follow-up queries after reviewing the first results. Its own example: “What are some good restaurants near me?” from someone in the San Francisco area may become the search query “top restaurants San Francisco.” If memory is on, saved details (such as a dietary preference) can be added to the rewritten query too.
The practical point: people ask in full sentences, but your page is matched against the shorter, more specific queries ChatGPT writes. A page that plainly states the service, the city, and the specifics (“emergency furnace repair in Burnaby, open Sundays”) gives those queries something to match.
Stage 3: Results come from OpenAI’s own index and from partners
This is the part most people get wrong, usually in one of two directions: “ChatGPT is just Bing” or “ChatGPT doesn’t use Bing at all.”
Does ChatGPT use Bing?
Partly, as one source among several. OpenAI’s help page says “ChatGPT search sometimes partners with other search providers” and sends them the rewritten queries. For details on how those providers process queries, it points readers to two privacy policies: Microsoft’s and Shopify’s. That suggests Microsoft’s search is among the inputs, though OpenAI doesn’t say how much weight it gets. It is one reason it is still worth making sure Bing can crawl and index your site, a point we made in One of Your Engines Runs on Bing.
But Bing is not the whole story, because OpenAI runs its own crawler and its own index.
OpenAI’s own crawler: OAI-SearchBot
OpenAI’s crawler documentation says OAI-SearchBot “is used to surface websites in search results in ChatGPT’s search features.” Sites that block it “will not be shown in ChatGPT search answers, though can still appear as navigational links.” OpenAI’s help page is just as direct: to be eligible, allow OAI-SearchBot and make sure your host or CDN allows traffic from OpenAI’s published IP addresses.
OpenAI uses three different agents, and only one of them decides whether you can appear in search:
| Agent | What it does | Does it affect ChatGPT search visibility? |
|---|---|---|
| OAI-SearchBot | Crawls pages automatically for ChatGPT’s search features | Yes. Block it and your pages won’t be shown in search answers, apart from possible navigational links |
| GPTBot | Crawls content that may be used to train OpenAI’s foundation models | No. OpenAI says each setting is independent, so you can block GPTBot and still allow OAI-SearchBot |
| ChatGPT-User | Visits a page when a user’s request in ChatGPT or a custom GPT calls for it | No. OpenAI says it “is not used to determine whether content may appear in Search” |
Source: OpenAI crawler overview, checked October 24, 2026. For every major AI crawler side by side, see our comparison of AI crawlers in robots.txt.
One more detail from OpenAI’s publisher FAQ: if a blocked page’s URL turns up through a third-party search provider or links on other pages, OpenAI may still show just the link and title. To keep a page out completely, it has to carry a noindex tag, and OAI-SearchBot has to be allowed to crawl it so it can see that tag.
How a page gets discovered, and what the Cloudflare pilot changed
A crawler can only fetch pages it knows about. Traditionally that means following links, and pages with few inbound links get found late or not at all. The Cloudflare pilot added another route.
Here is what the companies have actually said:
- What it is. A research pilot launched in July 2026 that tests whether signals from Cloudflare’s network can help OpenAI’s search crawler find pages worth indexing. Per Search Engine Journal’s coverage, Cloudflare’s July announcement named signals such as how recently a page was updated, the quality of traffic to a site, and what actually changed on a page.
- What it found. The pilot brought hundreds of millions of previously unknown pages into OpenAI’s search index. Cloudflare’s post argues that specific pages like a model’s manual or a restaurant’s hours “can be valuable even if few other sites link to them.”
- What it also does. The signals help the crawler skip URLs that don’t exist, sites that redirect everything to the homepage, and duplicate copies, and tell it when a page has changed enough to be worth another visit. Cloudflare and OpenAI are also testing signals that flag a single harmful page, such as a planted fake login form, without penalizing the whole domain.
- What it doesn’t do. “OpenAI then independently evaluates and ranks how those pages appear.” Being found doesn’t guarantee being cited. “A signal about a page doesn’t grant permission to access it,” so your crawler settings still apply. And the research uses metadata, not page content for model training.
- What we don’t know yet. As Search Engine Journal noted, the post doesn’t report how often the new pages were cited in ChatGPT or how much traffic they sent. Cloudflare has said it plans wider access later in 2026, but no date has been given.
If your site runs on Cloudflare, the post points to Cloudflare’s AI Crawl Control as the place to see and manage which AI crawlers reach your site. It is also where you would notice if OAI-SearchBot were being blocked by mistake.
Stage 4: Ranking, picking sources, and writing the answer
Once ChatGPT has candidate pages, OpenAI says it “ranks search results using multiple factors intended to help users find relevant, reliable information. Placement is not guaranteed.” OpenAI does not publish those factors, so anyone claiming a precise ChatGPT ranking formula is guessing.
What OpenAI does say:
- Answers come with citations that link to sources, plus a Sources panel with other relevant links.
- Citations can be wrong. OpenAI’s own help page warns that “search results and citations can be incomplete, outdated, or incorrect.”
- Local answers draw on more than websites. ChatGPT may share a general location with “trusted third-party providers that help provide more local information, such as nearby business listings.” For restaurants, it may show reservation availability from OpenTable, Resy, or Yelp.
That last point is why a business can be well indexed and still described badly: the answer may lean on a listing, a review site, or an aggregator instead of your own pages. We looked at that pattern in how ChatGPT recommends businesses.
What this means for a small business website
Each stage has a different point of failure, and they need different checks:
| Stage | What can go wrong | What you can check |
|---|---|---|
| 1. Decide to search | Model answers from old training data | Ask about your business with and without the Search tool and compare |
| 2. Rewrite the query | Your pages don’t plainly say what, where, and when | Service, city, hours and prices stated in text on the page, not only in images |
| 3. Index and partners | OAI-SearchBot blocked in robots.txt, or OpenAI’s IPs blocked by your host, CDN or bot protection | robots.txt allows OAI-SearchBot; your firewall or bot settings allow OpenAI’s published ranges; site indexed in Bing |
| 3. Discovery | Few links point to your important pages | Keep pages current; link to them from your homepage and sitemap |
| 4. Rank and cite | Listings or aggregators describe you differently than your site does | Same name, address, phone and hours everywhere; local business schema on your pages |
Two things you can measure for free. ChatGPT adds utm_source=chatgpt.com to referral links, according to OpenAI’s publisher FAQ, so ChatGPT visits can show up in Google Analytics. Some won’t, and we explained why in Your AI Traffic Is Hiding in Direct. And your server or CDN logs show whether OAI-SearchBot is visiting at all.
What neither shows is the answer itself: whether ChatGPT names you when a customer asks for a business like yours without using your name. For that you have to ask the engines. Beket AI’s free audit runs 25 prompts across 3 engines and shows which businesses get named. Beket AI Pro is $49 a month and re-audits daily, starting at 50 prompts across 3 engines, so you can see when the answer changes.
The short version
ChatGPT search is not one system. It decides whether to search, rewrites the question, pulls candidates from OpenAI’s own index and partner providers, ranks them by factors OpenAI doesn’t publish, and writes an answer that may cite you, a listing about you, or someone else. The Cloudflare pilot widens the front door for pages with few links. It doesn’t change what happens after a page is found.
Product details are from OpenAI’s help center and crawler documentation, Cloudflare’s blog, and Search Engine Journal, checked October 24, 2026. The Cloudflare pilot is a research program, so its scope may change.
Related posts
Bing AI Performance Report: How to See When Copilot Cites Your Site
How to set up and read the Bing Webmaster Tools AI Performance report, map grounding queries to pages, and use Topics Portfolio to find citation gaps.
llms.txt vs robots.txt vs Schema Markup: What Each One Does for AI Search
llms.txt, robots.txt and schema markup compared: what each file does, which AI systems read it, what Google says, and which one to fix first.
GPTBot vs OAI-SearchBot vs Google-Extended: Which AI Crawlers to Allow in robots.txt
AI crawlers do three different jobs: training, search, and user fetches. Compare every major bot and see which to allow in robots.txt to stay in AI answers.