At a glance: A growing share of B2B research is now done by AI agents — ChatGPT search, Perplexity, Gemini — fetching your pages on behalf of a human buyer. Agent-usable content has eight traits: an answer-first capsule near the top, question-shaped headings in a clean sequential hierarchy, tables for comparable facts, entity facts (name, positioning, claims) that are identical on every page, Article and FAQPage JSON-LD, a maintained llms.txt file, stable URLs, and no important content locked behind client-side JavaScript. This post practises each pattern it describes.
The next visitor to your pricing comparison or integration guide may not be a person. Increasingly it’s a machine sent by one — ChatGPT search, Perplexity or Gemini fetching your page in real time because a human buyer just asked a question your content might answer. The agent reads the page in seconds, extracts what it can parse, and either cites you in its answer or moves on.
That changes what “well-structured content” means. Search engines ranked pages; agents extract answers. This guide covers the structural side — how to write and mark up B2B content so a buying agent can parse, extract and cite it. Two adjacent layers are deliberately out of scope: the Model Context Protocol as a site-to-agent plumbing layer (we assess that separately) and making your forms and funnels operable by agents (a future guide).
Who is actually reading your B2B pages now?
The AI companies document this themselves, and their documentation describes three kinds of machine visitor.
Training crawlers (such as OpenAI’s GPTBot) collect content that may shape future models. Search-index bots build the retrieval layer: OpenAI says OAI-SearchBot is “used to surface websites in search results in ChatGPT’s search features”. On-demand user fetchers are the interesting ones for this post: they retrieve a specific page because a human just asked something. OpenAI’s documentation is explicit: “When users ask ChatGPT or a CustomGPT a question, it may visit a web page with a ChatGPT-User agent”, and ChatGPT-User is “not used for crawling the web in an automatic fashion”. Perplexity documents the same pattern for Perplexity-User, which “supports user actions within Perplexity”. On the Google side, the Google-Extended robots token controls Gemini training and grounding use; Google notes it “does not impact a site’s inclusion in Google Search”.
We see this first-hand. In Zian’s own server logs, we classify visits by user agent and watch on-demand fetchers like ChatGPT-User — alongside the search-index bots OAI-SearchBot and PerplexityBot — retrieve specific deep pages rather than the homepage: a single how-to guide, a comparison post, one FAQ entry. The pattern is unmistakably question-shaped — someone asked an assistant something, and the assistant came and read one precise page to answer it. (To surface this traffic in your own analytics, see our guide to measuring AI referral traffic in GA4 — worth doing, because AI search visitors convert better.)
What does an agent do with your page?
An agent working for a buyer runs a tight loop: fetch the page, parse its structure, extract the passages that answer the user’s question, then synthesise an answer — often citing the source. Every structural decision you make either helps or hurts that extraction step.
Two patterns matter most. First, answer-first capsules: a short, self-contained summary near the top of the page that directly answers the question the page exists to answer — like the box above. An agent skimming under time and token budgets can lift a capsule whole; it cannot lift an answer that only emerges across twelve paragraphs. Second, question-shaped headings in a clean hierarchy: headings that literally ask the buyer’s question, nested logically (one H1, H2s in sequence, no skipped levels). AirOps’ 2026 State of AI Search report (research compiled with search analyst Kevin Indig) found that “Sequential headings and rich schema correlate with 2.8× higher citation rates” and that “68.7% of pages cited in ChatGPT follow logical heading hierarchies”. Structure is not cosmetic to a parser; it is the map.
Reasoning-grade agents make the extraction step even more selective — they fetch more sources and quote less from each, a behaviour we unpacked in how ChatGPT’s thinking mode cites sources. The response is the same either way: make every section extractable on its own, claim before elaboration. If you’re building a broader AEO programme, our answer engine optimisation guide for SaaS is the companion piece.
Human-optimised vs agent-parseable: what changes?
Most B2B content is written to persuade a skimming human. Agents fail on different things than humans do. The table below contrasts the two patterns — not because you must abandon the left column, but because the right column has to be present as well.
| Element | Human-reader-optimised pattern | Agent-parseable pattern |
|---|---|---|
| Opening | Narrative hook; the answer arrives late for dramatic effect | Answer-first capsule near the top; the conclusion is extractable in one block |
| Headings | Clever or thematic (“Where the rubber meets the road”) | Question-shaped, literal, sequential hierarchy with no skipped levels |
| Key facts | Woven through prose across the page | Stated once, precisely, in a sentence or table row that stands alone |
| Comparisons | Discursive pros-and-cons paragraphs | An actual HTML table with consistent columns an agent can read row by row |
| Company claims | Rephrased freshly on every page | Identical entity facts everywhere — same name, positioning and figures verbatim |
| URLs | Whatever the CMS generates; slugs change with redesigns | Stable, descriptive, permanent — a moved page is a broken citation |
| Rendering | Client-side JavaScript apps; humans wait for hydration | Server-rendered HTML; most AI fetchers never execute your scripts |
| Machine layer | None, or whatever the SEO plugin defaults to | Article and FAQPage JSON-LD mirroring the visible content, plus llms.txt |
Why do entity facts have to match everywhere?
An agent rarely reads one page about you. It reconciles your homepage, a blog post, a third-party directory and whatever its index remembers — and inconsistency reads as unreliability. If your product is an autonomous AI sales agent platform on one page, an AI SDR tool on another and a conversational marketing suite in an old press release, the model has three weak entities instead of one strong one, and its answers about you will wobble accordingly. That wobble is measurable — AI answers to the same question already vary run to run (see our post on AI answer volatility) — and inconsistent source facts amplify it.
The practical discipline is a canonical fact sheet: one internal document holding your approved name, one-line positioning, product names and the few numbers you publish — every page drawing from it verbatim. We practise this at Zian: our agents’ track record is stated as “50,769+ qualified sales appointments set” everywhere the claim appears, in exactly those words, so a model meeting the same precise fact in many places can treat it as an attribute of the entity, not a stray claim on a stray page.
Consistency matters more, not less, because citation space is contested. Profound’s February 2026 study of ChatGPT citation sources — an analysis of more than 700,000 US ChatGPT conversations from October–December 2025 — found a “wide but shallow” citation economy: the top ten domains captured only about 12% of all citations, yet the distribution remained highly unequal, with cited conversations averaging roughly six unique sources. There is real room for specialist B2B sites in that long tail — we dug into who fills it in our post on AI citation source concentration. And because the buyer often never clicks through even when you are cited, the entity facts the agent extracts are your pitch — the dynamic we covered in zero-click AI search and brand visibility.
What belongs in the machine layer: JSON-LD, llms.txt, stable URLs, server-rendered HTML
JSON-LD. Article and FAQPage structured data restate your content in the format parsers were built for. The one rule that matters: the markup must mirror the visible page exactly — same questions, same answers. Schema that diverges from the rendered content is noise at best.
llms.txt. The llms.txt proposal — authored by Jeremy Howard in September 2024 and updated with a version 2 in August 2026 — is a markdown file at your site root that curates your key pages for language models: “We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content.” Its author is clear about when it helps: “Our expectation was that llms.txt would mainly be useful for inference rather than training” — that is, for exactly the on-demand, buyer-driven fetches this post is about. It is a proposal, not an enforced standard, but it costs an hour to publish; Zian keeps ours current, updated the day new content ships.
Stable URLs. Citations are links. Restructure your slugs and every answer that cited the old URL now recommends a 404. Treat published URLs as permanent interfaces; when a move is unavoidable, 301-redirect and keep the redirect forever.
Server-rendered HTML. Vercel’s December 2024 analysis of AI crawler behaviour on its network found that “none of the major AI crawlers currently render JavaScript” — including OpenAI’s and Anthropic’s fetchers and PerplexityBot — while “Google’s Gemini leverages Googlebot’s infrastructure, enabling full JavaScript rendering”. Capabilities evolve, but the safe assumption stands: content that only exists after client-side rendering is invisible to most agents deciding whether to cite you. Ship the substance in the initial HTML.
FAQ: structuring content for AI buying agents
What is agentic parsing?
Agentic parsing is how an AI agent — ChatGPT search, Perplexity, Gemini or a similar assistant acting for a human user — reads a web page it has fetched on demand: parsing the page’s structure, extracting the passages that answer the user’s question, and synthesising them into an answer that may cite the page. Writing for agentic parsing means structuring content so that extraction succeeds: answer-first capsules, question-shaped headings, tables, consistent facts and clean machine-readable markup.
Do AI agents respect robots.txt when fetching pages for a user?
Not always — and the vendors say so themselves. OpenAI’s crawler documentation states that for ChatGPT-User fetches, “Because these actions are initiated by a user, robots.txt rules may not apply.” Perplexity’s documentation for its Perplexity-User agent goes further: “Since a user requested the fetch, this fetcher generally ignores robots.txt rules.” Robots.txt remains the control for training crawlers and index bots, but on-demand user fetches are treated by vendors as the user browsing, not the bot crawling.
Can AI agents read JavaScript-rendered content?
Mostly not. Vercel’s December 2024 analysis of AI crawler traffic found that “none of the major AI crawlers currently render JavaScript”, with Gemini the exception because it uses Googlebot’s rendering infrastructure. Content that only appears after client-side rendering is effectively invisible to most AI fetchers, so important facts should be server-rendered into the initial HTML.
Does schema markup actually help AI citations?
Structure correlates strongly with being cited. AirOps’ 2026 State of AI Search report found that “Sequential headings and rich schema correlate with 2.8× higher citation rates” and that “68.7% of pages cited in ChatGPT follow logical heading hierarchies”. The consistent thread across the research is that clear structure — headings, standalone answers and markup that mirrors the visible content — is what parsers reward.
What is llms.txt and should a B2B site publish one?
llms.txt is an open proposal by Jeremy Howard, first published in September 2024 and updated in August 2026: “We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content.” It gives language models a curated, low-noise index of your most useful pages, aimed primarily at inference-time use — the moment an agent is researching on a user’s behalf. It is not an enforced standard, but adoption is widespread and the cost of publishing and maintaining one is trivial.
Buying agents are already reading your site for the people you want to reach — the structure above decides whether they can use what they find. If you’d rather your outbound was this systematic too, Zian’s autonomous AI sales agents handle prospecting, qualification and booking across phone, SMS, email and WhatsApp in 30+ languages. Apply For Partnership to join the waitlist.