Back to Blog
AI Discovery · Foundations

What is LLM SEO? Training corpus vs retrieval

The real definition is bigger than SEO. Two layers — training-time corpus inclusion and retrieval-time pipeline optimisation — decide whether an LLM knows your brand before any user types a word.

What is LLM SEO — training corpus vs retrieval diagram by WebFlur AI SEO Agency

LLM SEO is the practice of optimising both the training corpus that teaches a large language model (LLM — the AI system behind ChatGPT, Claude, and Gemini) what your brand means AND the retrieval pipeline it uses to recall and cite you at query time. It is broader than traditional SEO and overlaps heavily with GEO (Generative Engine Optimization — the broader practice of making content extractable by any generative AI engine). In WebFlur's 45-site B2B citation audit (Jan–Jul 2026), pages with any form of training-corpus presence were cited 3.2× more often by ChatGPT than retrieval-only pages — even when the retrieval-only pages had higher Google rankings.

In August 2026 a Series-B observability startup (62 people, Udyog Vihar Gurgaon) asked us why ChatGPT named three competitors but never them — even though they outranked all three on Google for "OpenTelemetry alternatives for Kubernetes". Their organic traffic was healthy. Their DR was 47. Their homepage was a crisp, benefit-first narrative.

But their training-corpus presence was zero. No Wikipedia entry. No Wikidata Q-ID. No arXiv paper. No README with their name in the top-100 OpenTelemetry GitHub repos. No mention in the Hacker News threads Common Crawl ingests heavily. On the retrieval side, their robots.txt allowed CCBot and GPTBot — so new content could in principle be learned — but the corpus had already been trained months earlier without them.

Their competitors were in all four places. That is the whole story. Google ranking optimises for a live index; LLM SEO optimises for what the model already knows before any query is run. The two pipelines reward different signals, and the P1 foundations pillar — AI SEO vs GEO vs AEO (AEO = Answer Engine Optimization — structuring pages so an AI assistant can lift a direct answer verbatim) — places LLM SEO as the training-data-first cousin of GEO. This post is the dedicated deep dive into that corpus-first layer.

What's the difference between LLM SEO at training time and LLM SEO at runtime?

Training-time LLM SEO shapes what the model memorises before it ships; retrieval-time LLM SEO shapes what the model fetches live when a user asks a question. They are two separate pipelines inside the same product, they reward different signals, and most teams only work the retrieval one. That's why "we rank on Google but nobody cites us in ChatGPT" is such a common complaint — it's a corpus problem, not a retrieval problem.

WebFlur audit — 45 B2B sites, 1,200+ citation events observed, Jan–Jul 2026
3.2×

Pages with any form of training-corpus presence (Wikipedia mention, Wikidata entity, GitHub README reference, arXiv citation, or named-outlet press from Reuters, CNBC, The Verge, Wired, or TechCrunch) were cited by ChatGPT at 3.2× the rate of structurally equivalent pages with Google-only authority. The corpus signal beat retrieval by a wider margin than any other variable in the dataset — including domain rating, schema richness, and content freshness.

Here's the clean way to think about it. The training corpus is everything the model has already read and compressed into weights — filtered Common Crawl snapshots, Wikipedia dumps, GitHub repos, arXiv papers, Reddit threads, licensed news archives. The retrieval pipeline is what the model looks up live when it does not know, is not sure, or is explicitly asked to search — Bing results for ChatGPT Search, Google results for Gemini, Sonar crawls for Perplexity.

Signal Training-time (corpus) Retrieval-time (runtime)
What matters mostWikipedia/Wikidata entity, arXiv, GitHub, named-outlet pressLive crawl access, schema richness, answer-first structure
Lag to visible impact90–180 days (next model refresh cycle)30–60 days (next crawl + index pass)
Which crawlersCCBot, GPTBot, Google-Extended, ClaudeBotBingbot, Googlebot, PerplexityBot, OAI-SearchBot
Where backlinks fitLink type matters (Wikipedia > 100 DR40 blogs)Link count/authority still passes the usual way
Reversibility if you break itLow — weights don't un-learn; wait for next refreshHigh — fix robots.txt or schema and recrawl
Corpus-time readiness checklist
  • Your brand has a Wikipedia article OR a Wikidata Q-ID (lower bar — aim for this first)
  • robots.txt allows CCBot (Common Crawl's crawler, which feeds most frontier training sets), GPTBot (OpenAI's training crawler), Google-Extended (Gemini's training-data signal), and ClaudeBot (Anthropic's crawler) — not just Googlebot
  • At least one of: an arXiv paper, a public GitHub repo with a non-trivial README, a named-outlet (Reuters / CNBC / Wired / TechCrunch / Fortune) feature
  • Your pages have entity-linked JSON-LD schema (the structured-data block inside your page's <script type="application/ld+json"> tag) whose sameAs, about[] and knowsAbout entries point to your Wikipedia + Wikidata records — without this, the model's grounding pass cannot connect your page to its concept graph, and ungrounded pages get filtered out during training-data selection
  • Your category and your product are described in identical language across your site, your Wikipedia/Wikidata entry, your LinkedIn, your Crunchbase — variance here splits the entity into "possibly two different companies" in the model's view
  • Your content exists in <p>/<li>/<table> HTML — not JavaScript-rendered DOM nodes, not PDFs, not image-only slides — so Common Crawl's HTML parser can lift it cleanly
  • You have a /llms.txt file (a plain-text index at your site root, like robots.txt but written for AI crawlers) that gives the training crawlers an entity-linked summary of your site — so they can map your most important pages without having to infer structure from your navigation menu

If you've worked on AI-assisted vs agentic SEO workflows, the training-corpus work is where agentic execution genuinely moves the needle — a human can't manually keep 400+ pages' entity schema in sync, but an agent can.

How is LLM SEO different from traditional SEO?

Traditional SEO targets Google's live index and ranks pages against each other; LLM SEO targets what the model memorises during training and what it retrieves at runtime — two separate pipelines that reward different signals. You can rank #1 on Google and still be invisible in ChatGPT if your corpus footprint is zero. The reverse is also true: brands can be heavily cited by LLMs for category queries while sitting on page 2 of Google — this happens most often when a company has strong Wikipedia / arXiv / GitHub presence but weak traditional backlink profiles.

The shift is less dramatic than headlines suggest, though. The underlying work has not been thrown out. Clean HTML still matters. Schema still matters. Internal linking still matters. Three things change: which sources earn you authority (Wikipedia + arXiv + GitHub + named outlets move up; DR 40 guest-post farms move way down), how long the feedback loop is (training-corpus changes surface at the next model refresh, not next week), and what the ranking unit is (a passage inside a page, not the page as a whole).

Josh Clark — a WebFlur-audit client CMO at a Series-B DevTools company — put it well in September 2026: "We've not been doing less SEO. We're doing more of it, in more places, for an audience that includes four models." The budget line didn't shrink. The work map got broader.

What are the six ranking signals that actually get you into an LLM's training corpus?

Across the 45-site WebFlur B2B audit (Jan–Jul 2026) and 1,200+ observed citation events, six corpus-side signals correlated most strongly with being named by ChatGPT, Claude, or Gemini for a category query. They stack — no single one is sufficient. Most B2B brands we audit have one of these, usually zero. The brands that get cited consistently have four or more.

  1. Wikipedia or Wikidata entity presence (strongest signal). Wikipedia is in every major LLM training corpus and is also the heaviest-weighted single source in Common Crawl. If you cannot pass Wikipedia's notability bar, start with Wikidata — it has a lower bar and still earns you a persistent entity ID that your schema can point to. The Wikidata Q-ID becomes the model's anchor for your brand.
  2. Named-outlet press coverage. Reuters, CNBC, The Verge, Wired, Fortune, TechCrunch, The Guardian, Bloomberg — these outlets have high crawl rates and high reliability weights in the training-data filtering stage. One Reuters profile is worth more than 50 trade-blog features here.
  3. arXiv, SSRN, or NBER paper authorship. If any part of your product is research-adjacent (ML, security, finance, biotech, logistics modelling), a working paper on an open archive is a disproportionately strong corpus signal. Technical sources are over-represented in training data by every lab that has published its data-mix.
  4. Open-source GitHub repo with a non-trivial README. Public GitHub is crawled aggressively by every lab building a coding assistant. A repo with a 1,500-word README that references your product, your category, and your use cases earns persistent memory.
  5. Entity-linked JSON-LD across your money pages. Every about[] and knowsAbout entry in your Article schema should be a Thing with sameAs pointing to Wikipedia and Wikidata. Without this, the model's grounding pass cannot connect your page to its concept graph — and ungroundable pages get down-weighted during training-data filtering.
  6. Reddit, Hacker News, and Stack Overflow mentions in the correct subreddit / tag. Common Crawl snapshots these heavily and they survive the content-quality filter. A thoughtful comment thread where real engineers debate your product in a relevant subreddit outperforms a sponsored post anywhere else.

None of these are shortcuts. All of them are earnable in a 6–12 month window if you treat them as a programme — which is what we build out inside the AI SEO strategy 2026 playbook, and which overlaps with the signals the P3 pillar — how Perplexity, Claude & ChatGPT cite differently — breaks down per-platform.

Does LLM SEO still need backlinks?

Backlinks still matter in LLM SEO, but their role shifts from count-based authority to source-based memorisation. A link from Wikipedia, a star on a GitHub repo that Common Crawl indexes heavily, or a citation inside an arXiv paper is worth more than 100 generic DR 40 backlinks — because those sources are inside the training data, not just inside Google's index. We've honestly not been able to find a single B2B brand in our audit that was cited consistently by ChatGPT without at least one high-trust corpus-side link.

The retrieval layer is more familiar. ChatGPT Search leans heavily on Bing, Gemini leans on Google, and Perplexity's Sonar does its own crawl. Here the usual link graph signals still work — link count, domain authority, anchor-text relevance. What's new is that these links have to live on pages the live-grounding pipeline will actually crawl at query time. Pages blocked by robots.txt, locked behind authentication, or shipped only inside PDFs don't register.

We can't fully predict which backlink combinations will earn a brand a Knowledge Graph entry at Google versus a Wikipedia entry versus neither — the models' training-data filtering is not fully documented. What we can say from the audit is: the brands that compounded both layers (corpus-side links + retrieval-side links) consistently outperformed brands that over-invested in one.

How do you measure AI visibility — and what breaks when you measure it wrong?

AI visibility is the share of category-level buyer queries in which your brand appears as a named entity in an AI response (ChatGPT, Claude, Perplexity, Gemini, Google AI Mode) — measured consistently, in incognito, across geos, over time. The two most common mistakes are measuring once and declaring victory, and measuring only in one assistant. Our audit panel runs 20–30 category queries weekly across four assistants in two geos (US and IN) because the variance between a single-query run and a 30-query panel is enormous.

If you're new to AI-visibility measurement, the P4 pillar — how to track AI SEO — is the fuller walkthrough. The short version: use incognito. Log the exact prompt. Record the citation position (named in body / mentioned in footnotes / absent). Repeat weekly for at least 60 days before drawing a trend line.

The failure mode we see most often: a founder runs a query twice, gets cited once, and declares the problem solved. LLM responses are non-deterministic. A single-query win is noise. Even brands with excellent corpus presence have week-over-week citation variance of 15–25%. If your sample size is 1, you are measuring randomness.

Sources & further reading

Want to see which training-corpus signals your brand is missing?

Get your AI audit →
Pankaj, Founder of WebFlur
Written by
Pankaj
Founder, WebFlur — AI SEO & Agentic Presence Engineer

Pankaj has spent a decade building SEO infrastructure for B2B companies. He co-founded WebFlur to focus on the training-corpus signals that decide whether ChatGPT, Claude, Perplexity, and Gemini name a brand — and the retrieval-pipeline work that keeps that visibility live. Runs the 45-site B2B citation audit referenced here. Connect on LinkedIn.

Frequently asked questions

LLM SEO is the work of making a large language model like ChatGPT, Claude, or Gemini both learn about your brand during training AND recall it accurately at query time. It has two layers: corpus-time (what the model memorises before it ships) and retrieval-time (what the model fetches live when a user asks). Most teams only work the retrieval layer and wonder why they stay invisible.
They overlap almost completely, with one shift in emphasis. GEO (Generative Engine Optimization) is the umbrella term for making content extractable by generative AI. LLM SEO uses the same mechanics but stresses the training-corpus side — Wikipedia, arXiv, GitHub, Common Crawl — because that is what decides what the model knows before any live retrieval happens. Pick one label and be consistent; the playbook is the same.
Traditional SEO targets Google's live index and ranks pages against each other. LLM SEO targets what a model memorises during training and what it retrieves at runtime — two different pipelines that reward different signals. Backlinks still help but matter less; corpus presence (Wikipedia, GitHub, arXiv) and entity-linked schema matter more. You can rank #1 on Google and still be invisible in ChatGPT if your corpus footprint is zero.
Backlinks still matter, but their role changes. For the retrieval layer, links from pages a model crawls at query time (via Bing, Google, or dedicated LLM crawlers) still pass authority. For the corpus layer, the kind of link matters more than the count — a mention on Wikipedia, a citation in an arXiv paper, or a star on a well-crawled GitHub repo is worth more than 100 generic DR 40 backlinks because those sources sit inside the training data.
Retrieval-pipeline gains (RAG — retrieval-augmented generation, where the model fetches live pages mid-answer and cites them) show up in 30–60 days — this is where ChatGPT Search, Perplexity, and Google AI Mode re-index your pages. Corpus gains show up at the next model refresh cycle, usually 90–180 days for frontier labs. WebFlur's audit data shows the median B2B brand needs both layers running in parallel to shift from invisible to consistently cited.
Common Crawl snapshots (the public web crawl most frontier labs filter and ingest), Wikipedia and Wikidata, GitHub public repos and READMEs, arXiv papers, Reddit threads, Hacker News comments, and licensed news archives. Pages that get crawled by Common Crawl but have thin content or no entity anchoring frequently get filtered out during training. Being in the crawl is not the same as being learned from it.