How to Structure a Webpage When LLMs Can Extract Only 25–30 KB of Data

What limits do AI systems actually apply?

The picture is consistent even without an exact byte count. Assistants do not read a page the way a person does. They fetch it, strip the markup, split the text into passages, score the passages against the question, and keep a small number of short extracts within a fixed budget.

Why does the 2MB limit matter for the agentic web?

Google's March 2026 explanation of Googlebot is explicit: it fetches up to 2 MB of any URL except PDFs, stops at that point, and passes the partial file to indexing and rendering as if it were complete. Anything beyond is never indexed. Referenced CSS and JavaScript files are fetched separately, each with its own 2 MB limit.

That matters beyond classic search. Google's AI Overviews and AI Mode build on the same index, so content past the cutoff cannot ground them. Google notes most pages never come close, but warns that inline base64 images, large blocks of inline CSS or JavaScript, or megabytes of menus before the content can push the real text or structured data past the limit.

For agents more generally, the lesson is proportion. The same bloat that risks a hard cutoff for Googlebot also makes pages slow and costly for agents that fetch on demand, and wastes their limited context on navigation and code. Shopify-specific causes are covered in how bloated Shopify themes limit AI visibility.

Why are only the first 500 words of my page appearing in AI citations?

Usually because that is where the extractable answers are, and because of how selection works:

• Answer-first pages win the budget. Surfer found citation frequency peaks in the first 100 words and falls sharply through the first 500. Its share of citations in the first 100 words rose from 20% in its 2025 sample to 38% in 2026.
• Extraction is verbatim and short. With extracts of around 15 words and a shared budget, a system takes the sentences that answer the question most directly, which on most pages sit near the top.
• Later content may be harder to reach. Content in tabs or accordions loaded by JavaScript, or appearing after long code in the HTML source, may not be read at all by assistants that do not run JavaScript. A 2026 test by Advanced Web Ranking found the major US assistants answered from the raw HTML, not the JavaScript-rendered page.
• Structured data is not a substitute. An October 2025 test by searchVIU found that systems fetching a page live did not read a price that existed only in JSON-LD.

If your deeper sections never get cited, check whether they answer a distinct question clearly, and whether they appear in the HTML at all.

How should you structure a page?

1. Lead with the answer. The first paragraph should answer the page's main question in 40 to 100 words, with the key facts, numbers and names.
2. Make every section stand alone. Use a question as the heading, answer it in the first sentence, and name the subject instead of relying on “it” or “this”, so an extracted sentence still makes sense.
3. Put facts in sentences and tables. Prices, dimensions, dates and comparisons belong in visible text, not only in structured data or images.
4. Keep the source order clean. Main content should come early in the HTML, not after mega-menus, inline scripts and embedded styles.
5. Move code out of the HTML. Use external CSS and JavaScript files, avoid inline base64 images, and render key content on the server.
6. Cut filler. Scene-setting introductions and repeated summaries use budget without adding extractable facts.
7. Split very long pages into focused pages when sections answer unrelated questions.

How passages are split and chosen is covered in how content chunking affects what LLMs retrieve. Markdown versions can reduce the markup agents have to strip, as covered in how to serve Markdown to AI agents, but they do not replace a well-structured HTML page.

How can you check your own pages?

• Measure HTML size with your browser's network panel or a crawler, and look at what comes before the main content.
• View the page source, not the inspector, and confirm the answer and key facts appear near the top without JavaScript.
• Compare citations with position: note which sentences AI answers quote and where they sit on the page.
• Check logs for AI fetchers and the response size they received, as covered in how to tell whether LLMs use your content.

We have not measured how many kilobytes each AI assistant reads per page, and we could not verify the 25–30 KB figure. This page is based on Google's documentation of Googlebot's limits and published studies by DEJAN, Surfer, searchVIU and Advanced Web Ranking.

What might this look like in practice?

Consider an illustrative home insurance comparison site, Hearthcover, whose 4,000-word guide opened with three paragraphs about the history of home insurance. AI Overviews quoted only a definition from near the top, and the page source showed 900 KB of inline scripts and a mega-menu before the first heading. The team moved the scripts to external files, rewrote the opening as a 70-word answer with average premiums and the three factors that change them, and turned each section heading into a question answered in its first sentence. Within two months, AI answers began quoting the cost table and the section on flood cover.

What do people ask about page length and AI extraction?

Do LLMs really read only 25–30 KB of a page?

The figure is unverified. The limits AI providers do publish are far larger, such as 2 MB per HTML file for Googlebot and 34 MB per URL for the Gemini API's URL context tool, and none says how much of a page an assistant actually quotes. Documented behaviour favours short, early, extractable passages.

Does Googlebot really stop at 2 MB?

Yes. Google says Googlebot fetches the first 2 MB of an HTML file and ignores the rest. PDFs have a 64 MB limit.

Should I make all my pages short?

No. Make them dense and well ordered. Long pages work when each section answers a clear question and the most important answer comes first.

Does structured data help AI extraction?

It helps Google Search, but live-fetch AI systems tested so far did not read facts that existed only in JSON-LD. Put key facts in visible text too.