How Do I Make JavaScript-Rendered Pages Readable to AI Crawlers?

Which AI crawlers run JavaScript?

The same study found some crawlers do read text embedded in the initial HTML inside script tags, such as JSON data for hydration. That is not a dependable way to publish content.

Can AI search engines read JavaScript-rendered website content?

Some can, most cannot. Google's AI features can, because they draw on Google's index, which Googlebot builds by rendering pages. ChatGPT, Claude and Perplexity generally cannot, because their crawlers read only the raw HTML response. So a page whose main text is loaded by JavaScript may appear in AI Overviews and still be invisible to ChatGPT. A quick way to see what non-rendering crawlers see is to view the page source in your browser, not the inspected page, and search for a key sentence.

Does Gemini 2.5 read rendered HTML without JavaScript, and should you prerender pages?

Gemini grounded in Google Search uses Google's index, which is built from rendered pages, so JavaScript content that Googlebot can render is available to it. Gemini 2.5 has since been followed by newer models, but this has not changed. Gemini's URL context tool, which reads pages a user supplies, first looks for a page in Google's index cache and falls back to a live fetch only when the page is not there. Prerendering is not needed for Gemini if Googlebot renders your pages correctly. It is still worth doing for every other assistant. Server-side rendering also makes pages faster, which Google notes helps both users and crawlers.

Does Applebot render pages more fully than other AI crawlers?

Yes. Apple's documentation says Applebot may render your website's content within a browser. It warns that if JavaScript, CSS or other resources are blocked in robots.txt, it may not render properly. The Vercel and MERJ study also found Applebot rendered JavaScript. So Apple's search features in Siri, Spotlight and Safari can see content that ChatGPT and Perplexity miss. Applebot-Extended is separate. It is a robots.txt setting that controls whether content is used to train Apple's generative models, not a separate crawler.

How do you fix JavaScript-rendered pages?

Does server-side rendering help AI crawlers read website content?

Yes, it is the most reliable fix. With server-side rendering, the server sends complete HTML for each page, so every crawler sees the content whether or not it runs scripts. Static generation, where pages are built ahead of time, has the same effect and is faster still. Google's own guidance says server-side or pre-rendering is still a great idea because not all bots can run JavaScript. For pages that change often, such as prices and stock, render on each request or rebuild frequently, so crawlers do not see stale data.

Can AI crawlers read a React single-page app without prerendering?

Usually not. A typical client-side React app sends an almost empty HTML shell and builds the page in the browser. Non-rendering crawlers see the shell, often little more than a title and an empty root element. Googlebot will usually render it, but other assistants will not. Unless the app uses server-side rendering, static generation or prerendering, most of its content is invisible to ChatGPT, Claude and Perplexity.

What React-specific changes can improve AI search visibility?

• Use a framework with server rendering, such as Next.js or Remix, or static generation for content pages.
• Fetch key content on the server, not in effects that run after the page loads in the browser.
• Use real links: <a href> elements with full URLs, not click handlers that change the view. Google says it can only discover links that are <a> elements with an href attribute.
• Use the History API, not URL fragments, so each view has its own address.
• Render titles, descriptions and structured data on the server.
• Return proper 404 status codes for missing pages, rather than an empty view with a 200 status, which Google treats as a soft 404.
• Keep important text out of components that load only on click, such as modals and tabs that fetch content when opened.

Prerender.io for AI crawler visibility: is it a good stop-gap for dynamic content?

Yes, as a stop-gap. Prerendering services load your pages in a browser, save the rendered HTML and serve it to crawlers they recognise by user agent. Prerender.io publishes a list of AI crawler user agents to add, from GPTBot and ClaudeBot onwards. Google treats this approach, which it calls dynamic rendering, as a workaround rather than a long-term solution. It does not generally consider it cloaking, as long as the content matches what users see. Watch for three risks. Cached pages can go stale, showing crawlers old prices or stock. New AI user agents may be missing from your configuration. And the service adds cost and a dependency. Plan to move important pages to server rendering.

How do specific patterns affect AI crawlers?

Does lazy loading hide product specifications from AI crawlers?

It can. Native lazy loading of images with loading="lazy" is fine. The problem is text loaded by JavaScript only when a user scrolls or clicks, such as specifications fetched when a tab opens. Non-rendering crawlers never see it. Google advises loading content when it becomes visible in the viewport, without relying on scrolling or clicking, because Google Search does not interact with pages. For AI crawlers, the safest approach is to include specifications in the initial HTML, even if they sit in a collapsed section styled with CSS.

Pagination versus infinite scroll for Google AI Overviews and Perplexity

Pagination is safer. With infinite scroll, only the first batch of items is in the initial HTML, and later items load as the user scrolls, which crawlers do not do. Google recommends that infinite scroll be backed by paginated loading. Give each chunk its own persistent URL, link the pages in sequence, and update the displayed URL as users scroll. Perplexity's crawler, which does not run JavaScript, needs the same paginated URLs even more. Category pages that list products should link to every product through plain paginated pages.

How to make iframe content accessible to AI crawlers

Content inside an iframe comes from a separate URL. Google may index it as part of the page that embeds it, and offers an indexifembedded robots tag to control this. Do not assume AI crawlers follow an iframe's source into the page that embeds it; most fetch only the URL they were given. If content matters, such as a price table, a booking widget's key details or an embedded document, put the text on the page itself. If the iframe must stay, make sure its source URL is crawlable, has its own clear title and content, and is linked from somewhere visible.

Developers validating Google AI Overviews access to shadow DOM content

Google says it supports web components and, when rendering a page, flattens shadow DOM and light DOM content. So content in shadow DOM that renders correctly can be available to Google's AI features. Check it with the URL Inspection tool's rendered HTML in Search Console, and confirm the text appears. For other AI crawlers, shadow DOM content created by JavaScript is invisible, as with any client-rendered content. Declarative shadow DOM, written in the HTML itself, puts the markup in the server response, but crawlers may not all extract it. Key text is safest in the light DOM.

How can a developer make a JavaScript store locator crawlable for AI search?

A map-based store locator that loads locations through an API is invisible to non-rendering crawlers and hard for any crawler to navigate. Fix it by publishing a static page for each location, with name, address, phone, hours and services in HTML and LocalBusiness structured data. Add a crawlable index page that lists every location with <a href> links, grouped by region. Include location pages in your sitemap. Keep the interactive map for users, but make the location pages the source of truth for assistants answering "nearest [brand] to [place]".

Can AI bots read content inserted by a client-side A/B testing platform?

Non-rendering AI bots see only the original page, because client-side testing tools change the page in the browser. Googlebot may see a variant if the script runs during rendering. This matters when a test changes facts such as prices, offers or product claims. Crawlers may then describe a version most users no longer see, or one some users never see. Keep facts consistent across variants, or test them server-side with care. Google's testing guidance says not to show different content to Googlebot than to users and to run tests only as long as necessary. Where variants use separate URLs, it recommends canonical tags and temporary redirects.

Which pages should you fix first?

Prioritise by what assistants need to answer buyer questions about you:

1. Pricing and plan pages, since wrong or missing prices cause the most damaging errors.
2. Product and service pages, with specifications, features and limits.
3. Location and contact pages, with addresses, hours and phone numbers.
4. Comparison, integration and use-case pages that answer choosing questions.
5. Guides that already rank in Google, which assistants may retrieve if they can read them.

Account areas, dashboards and internal tools rarely need attention, since crawlers should not be reading them anyway. A site can often fix its twenty most important templates and cover most of what matters.

How do you test what AI crawlers see?

1. View the page source in a browser and search for key sentences. If they are missing, non-rendering crawlers cannot see them.
2. Fetch the page with an AI crawler's user agent using a command-line tool, and compare the text with what the browser shows.
3. Disable JavaScript in the browser and reload the page.
4. Use Search Console's URL Inspection to see Google's rendered HTML.
5. Check server logs for AI crawler requests, and confirm they receive a 200 response with full content, not an error or a challenge page.

Related access problems, such as firewalls and robots.txt, are covered in how to use robots.txt and a WAF to allow useful AI bots and block scrapers. Documents that crawlers struggle with are covered in how to make PDF content accessible to AI crawlers.

What do people ask about JavaScript and AI crawlers?

Is structured data enough if the visible content is rendered by JavaScript?

No. Tests by SearchVIU found that ChatGPT, Claude, Perplexity and Google's AI features, when fetching a page directly, used the visible HTML text rather than the JSON-LD. Put the content itself in the HTML.

Will Google penalise prerendering for bots?

Not if the prerendered content matches what users see. Google calls dynamic rendering a workaround, not a violation.

Do I need to rebuild my whole site?

Usually not. Start with the pages that matter most for AI answers: product, pricing, service, location and key guide pages.

How quickly will AI crawlers see the fixed content?

After their next visit. Well-linked pages are usually recrawled within days to weeks.