Most advice about writing for AI search focuses on what to say. This guide is about something more mechanical: how retrieval pipelines physically slice your page before an AI model ever reads it, and what that slicing means for the way you structure sections, headings and tables. The numbers here come from vendor documentation, not folklore — and where a vendor documents nothing, we say so.
What a retrieval chunk actually is
A retrieval chunk is a contiguous slice of a document — measured in tokens, not words or paragraphs — that a retrieval system stores and searches as a single unit. When a document enters a retrieval pipeline, it is split into these slices, each slice is converted into an embedding (a numerical representation of its meaning), and the embeddings are indexed. When a user asks a question, the system embeds the question, finds the chunks whose embeddings sit closest to it, and hands only those chunks to the model that writes the answer.
The critical consequence: the model answering the question usually never sees your whole page. It sees a handful of isolated slices, competing against slices from every other document in the index. Whatever context lived outside a slice — the company name in your intro, the definition two sections up, the caveat in the previous paragraph — is gone unless the slice carries it internally.
The documented numbers: 800 tokens, 400-token overlap
OpenAI’s retrieval documentation is the clearest public reference for how a major vendor chunks by default. For its vector stores — the storage layer behind the file search tool — the docs state that by default max_chunk_size_tokens is set to 800 and chunk_overlap_tokens is set to 400. In other words: slices of up to 800 tokens, with each consecutive pair sharing 400 tokens of overlap. The same page documents that a file search response contains 10 results maximum by default, configurable up to 50 via max_num_results.
Two practical translations of those numbers:
- 800 tokens is roughly 600 words of English prose (using the common rule of thumb of about three-quarters of a word per token). That is a long-ish blog section: an h2 plus four to six paragraphs, or an h2 with a short table.
- The 400-token overlap means every passage effectively appears in two chunks, each seeing about 300 words of surrounding context on one side. Overlap softens bad splits — it does not rescue a passage whose necessary context sits 1,000 words away.
What other vendors document
It is worth being precise about who documents what, because most “chunking best practice” articles blur this badly.
| Vendor | Product | Documented default | Configurable? |
|---|---|---|---|
| OpenAI | Vector stores / file search | 800 tokens per chunk, 400-token overlap | Yes — max_chunk_size_tokens, chunk_overlap_tokens |
| Gemini API File Search | Automatic chunking documented; default numbers not published on the docs page | Yes — chunking_config with max_tokens_per_chunk, max_overlap_tokens | |
| Anthropic | No hosted vector store | None — chunking is the developer’s job; Anthropic’s engineering research discusses chunk-context loss (see below) | N/A |
Google’s Gemini File Search documentation confirms the same architecture — files are “automatically broken down into chunks, embedded, indexed” — and exposes the same two knobs (chunk size and overlap, in tokens), but does not publish its default values on the documentation page. Anthropic operates no hosted retrieval store at all, so it documents no chunking defaults; if you build retrieval over Claude, you choose your own chunker.
An honest caveat: these are file-retrieval defaults, not ChatGPT search internals
This matters enough to state plainly. The 800/400 numbers are documented for OpenAI’s vector store and file search product — the pipeline developers use to give models access to uploaded documents. OpenAI has not published the chunking parameters that ChatGPT’s live web search uses when it reads your blog post. Treating 800/400 as a confirmed ChatGPT-search constant would be overclaiming.
The bridge is inferential, but it is a reasonable inference: 800/400 is what OpenAI chose as a sensible general-purpose default when it had to pick one for its own retrieval product, and every retrieval pipeline — vendor-hosted or homegrown, and including the enterprise RAG systems that increasingly sit between your content and a buyer’s AI assistant — faces the same physics. Slices in the hundreds-of-tokens range, retrieved in isolation, are the operating environment your content lives in. Writing sections that survive an 800-token guillotine is robust across all of them, whatever the exact blade width.
Why self-contained chunks win: the “as mentioned above” problem
Anthropic’s engineering post on contextual retrieval gives the canonical example. Imagine a financial document chunked traditionally, producing the slice: “The company’s revenue grew by 3% over the previous quarter.” Which company? Which quarter? The chunk does not say — so a query about a specific company’s recent growth may never retrieve it, and a model handed the chunk cannot safely cite it. Anthropic’s proposed fix, prepending explanatory context to every chunk before embedding, reduced the top-20-chunk retrieval failure rate by 35% in their tests (from 5.7% to 3.7%), and by 49% when combined with contextual keyword matching.
Here is the part that matters for writers: vendors can only apply that kind of repair inside their own pipelines. Nobody is running contextual enrichment over your blog post before ChatGPT search or an enterprise RAG system slices it. On the open web, the only party who can put the context inside the chunk is the author. Every retrieval failure Anthropic measured is a failure you can pre-empt at the keyboard.
Concretely, a chunk-hostile passage and a chunk-friendly one can carry identical information:
| Pattern | Chunk-hostile | Chunk-friendly |
|---|---|---|
| Referencing earlier content | “As mentioned above, this approach also works for SMS.” | “Consent-first outreach also works for SMS.” |
| Naming the subject | “It handles this automatically.” (three paragraphs after naming the product) | “SmartReach AI handles follow-up timing automatically.” |
| Attributing a figure | “The default is 800 tokens.” (source named in a previous section) | “OpenAI’s retrieval docs set the default at 800 tokens per chunk.” |
| Definitions | Definition in the intro only; later sections assume it | One-line re-definition at the top of each major section |
| Headings | “More details” / “Going further” | “How 400-token overlap affects section length” |
This is the same discipline we described in our guide to entity consistency in AI search — use the same full name for your organisation and products everywhere — but applied at paragraph scale: the entity must recur often enough that any 600-word window contains it.
If your pipeline for demand doesn’t yet assume AI intermediaries at all, that’s the bigger gap. Zian AI’s digital team agents — from the Outbound Appointment Setter to the 24/7 Customer Support Agent — work from the same principle this article describes: answers that carry their own context, whatever channel they travel through. Apply For Partnership
A practical checklist for chunk-aware page structure
1. Treat every h2 section as a potential orphan
Write each major section so that, pasted alone into an email, it would still make sense: who is speaking, what is claimed, where the claim comes from. The 800-token default means an h2 section of roughly 400–600 words plus its heading will often travel as one chunk — or as two overlapping ones, thanks to the 400-token overlap.
2. Front-load the answer inside each section, not just the page
Page-level front-loading — answer capsule first, evidence after — is covered in our post on front-loading and AI citation position. Chunk-awareness extends the same rule downward: the first sentence under each heading should state that section’s conclusion, because a chunk boundary may fall anywhere after it.
3. Make headings self-describing
Most chunkers split on structural boundaries where they can, and a heading frequently ends up as the first line of a chunk. “How 800-token chunks affect section length” tells a retrieval system what the following tokens are about; “Digging deeper” tells it nothing. Heading hygiene is free retrieval metadata.
4. Re-name entities relentlessly
Pronouns and back-references are the “3% revenue growth” trap. Once per paragraph-cluster, use the full name: the organisation, the product, the law, the study. It reads slightly redundant to a human skimming the whole page; it reads perfectly to a model holding one slice.
5. Keep tables compact and captioned in prose
Tables are excellent for chunk survival when they are small: a 10-row comparison table with headers fits comfortably inside one 800-token chunk, headers and all, and arrives as pure structured signal. A 60-row table gets split mid-body, producing chunks of orphaned cells with no header row — worse than prose. Rule of thumb: keep tables under roughly 400 tokens, and precede each with one sentence of prose stating what the table shows, so the sentence and the header row travel together.
6. Put claim + source in the same sentence
“OpenAI’s retrieval documentation sets the default chunk size at 800 tokens” is a citable unit. A number in one paragraph and its attribution two paragraphs later can land in different chunks, leaving the number unsourced where it matters. This also aligns with how autonomous agents read pages when acting for a buyer — covered in our guide to writing for agentic parsing.
Sources & ownership
| Figure / claim | Owning organisation | Owner URL |
|---|---|---|
| Default max_chunk_size_tokens = 800, chunk_overlap_tokens = 400 for vector stores; file search returns 10 results by default, up to 50 via max_num_results | OpenAI | developers.openai.com/api/docs/guides/retrieval |
| Gemini File Search chunks files automatically; chunking configurable via chunking_config (max_tokens_per_chunk, max_overlap_tokens); default values not published on the page | ai.google.dev/gemini-api/docs/file-search | |
| Context-free chunk example (“The company’s revenue grew by 3% over the previous quarter.”); contextual embeddings cut top-20-chunk retrieval failure rate by 35% (5.7% → 3.7%), 49% (5.7% → 2.9%) combined with contextual BM25 | Anthropic | anthropic.com/engineering/contextual-retrieval |
| ~800 tokens ≈ ~600 English words | Rule of thumb (¾ word per token), stated as such — not a vendor figure | — |
FAQ
What is a retrieval chunk?
A retrieval chunk is a contiguous slice of a document, measured in tokens, that a retrieval system embeds, indexes and retrieves as one unit. When an AI system answers a question from indexed content, it typically sees only the retrieved chunks — not the full page — so any context outside the chunk is invisible to it.
Is 800 tokens an official industry standard for chunk size?
No. It is the documented default for one vendor’s product: OpenAI’s retrieval documentation states that for its vector stores, max_chunk_size_tokens defaults to 800 and chunk_overlap_tokens defaults to 400 (see OpenAI’s retrieval guide). Google documents that Gemini File Search chunks files automatically with configurable size and overlap, but does not publish its default numbers, and Anthropic operates no hosted vector store at all. There is no cross-industry standard — 800/400 is simply the best-documented public default.
Does ChatGPT’s web search split my page into 800-token chunks?
Nobody outside OpenAI knows, and OpenAI has not published the chunking parameters ChatGPT search uses on live web pages. The 800/400 default is documented only for the vector store and file search product. The practical takeaway is directional, not literal: every retrieval pipeline slices pages into isolated passages of a few hundred tokens, so sections written to survive an 800-token cut are robust regardless of the exact number.
How long is 800 tokens in ordinary writing?
Roughly 600 words of English prose, using the common rule of thumb of about three-quarters of a word per token. That is approximately one substantial h2 section — a heading plus four to six paragraphs, or a heading plus a compact table.
Do I need to rewrite every existing page to be chunk-aware?
No — prioritise. Start with the pages you most want cited by AI systems: definition pages, statistics pages and comparison pages. On those, fix the highest-impact patterns first: remove “as mentioned above” back-references, restate the organisation or product name in each section, put each key figure and its source in the same sentence, and rewrite vague headings. New content should simply be written chunk-aware from the start.
Structuring pages for machine readers is one half of the equation; having an AI sales team that acts on the demand those pages generate is the other. Zian AI is currently onboarding partners from its waitlist. Apply For Partnership