Resources · Technical Access for AI Crawlers

How Can Brands Make Their Websites Readable to AI Agents?

Brands can make their websites readable to AI agents by building pages that both a screen reader and a script-free crawler can understand. Put key facts in the server-rendered HTML. Use native HTML elements for buttons, links and forms, with visible labels and clear headings. Keep flows predictable, and avoid interactions that only work with a mouse. AI agents, which act on a user's behalf rather than just answering questions, read pages in three ways: as screenshots, as the HTML document, and as the accessibility tree. Many combine them. OpenAI said its agent in the Atlas browser used ARIA roles and labels to understand pages, and Google's Lighthouse now has an experimental category for agentic browsing. Good accessibility is most of the work.

How do AI agents read a website’s DOM and accessibility tree?

There are three main approaches, often used together:

Approach What the agent sees Example
Screenshots Pixels, interpreted by a vision model, with a virtual mouse and keyboard OpenAI's Computer-Using Agent processes raw pixel data and uses a virtual mouse and keyboard
The DOM (the page's HTML structure) Elements, text, attributes and links, as the browser built them Agents running in a browser can read the document after scripts run
The accessibility tree Roles (button, link, textbox), accessible names, states and values, as assistive technology sees them Microsoft's Playwright MCP server operates on the accessibility tree rather than pixels; OpenAI said its agent in the Atlas browser, retired in August 2026, used ARIA labels and roles

The accessibility tree is a simplified view of the page that browsers build for screen readers. Each interactive element has a role, a name and a state. When an agent reads it, a button labelled "Add to basket" is a clear, actionable target. A div with a click handler and an icon is not, unless it has been given a role and a name. Search crawlers that feed AI answers are different again. Most do not run JavaScript, so they see only the HTML your server sends.

What should you fix first?

  1. Key facts in the initial HTML: product names, prices, availability, hours, addresses and policies, not loaded only by script.
  2. Native elements: real <button>, <a href>, <select> and <input> elements, not styled divs with click handlers.
  3. Labels for every form field, connected to the field, not just placeholder text.
  4. Accessible names for icon buttons, such as "Close", "Search" or "Add to basket".
  5. A logical heading structure that describes each section.
  6. Predictable flows: no content that appears only on hover, no drag-only controls, and clear error messages in text.
  7. Stable, descriptive URLs for products, categories and steps, so an agent can return to a page.
  8. A stable layout: controls such as "Add to basket" stay in the same place and do not jump as images, banners or prices load.

The W3C's guidance on ARIA puts native HTML first. If a native element already has the semantics you need, use it rather than adding an ARIA role to something else. Accessibility specialists have warned that adding ARIA carelessly to please AI agents can make pages worse for disabled users. Fix the HTML first, and use ARIA to fill gaps.

Which pages matter most to agents?

Agents act on specific tasks, so a handful of page types carry most of the weight:

  • Product and service pages, where the agent confirms it has the right item: name, variant, price, availability and key specifications, all in text and in matching structured data.
  • Pricing and plan pages, where it compares options: a plain table rather than toggles that hide the other plans.
  • Policy pages: delivery, returns, cancellation and warranty, stated plainly with numbers and conditions, since agents check them before committing.
  • Search and category pages, with filters that are real links or form controls, and results that load without endless scrolling.
  • Account, basket and checkout pages, where every step must be operable without a mouse.
  • Contact and booking pages, with the options and times shown as text.

Review these first. Most sites have hundreds of pages, but an agent completing a purchase or a booking touches perhaps six, and a single unusable step ends the task.

What about checkout, booking and other flows?

Agents that complete tasks struggle with the same things that frustrate people using keyboards and screen readers. Watch for:

  • CAPTCHAs and bot checks on every step, which may block legitimate agents along with abusive bots.
  • Pop-ups and interstitials that cover the page without a clear, labelled close button.
  • Date pickers and custom dropdowns that cannot be operated by keyboard or typed into.
  • Multi-step forms that lose data when a step is revisited.
  • Prices and fees that appear only at the last step.

How to structure booking flows specifically is covered in how to structure a website so AI agents can complete bookings.

How can you test agent readability?

  • Lighthouse's Agentic Browsing category: introduced in Lighthouse 13.3, it runs deterministic audits of how well a site is built for machine interaction. It uses a subset of accessibility audits important for agents (accessible names, labels, visibility), a layout-shift check, an optional llms.txt check (a missing file is marked not applicable) and checks for the proposed WebMCP standard. It needs Chrome 150 or later, and PageSpeed Insights may still run an older Lighthouse. Google marks it as experimental and based on proposed standards.
  • The accessibility tree in browser developer tools: check that each important control has a sensible role and name.
  • A keyboard-only run: complete your main task, such as buying, booking or requesting a quote, without a mouse.
  • A script-free fetch: request key pages without JavaScript and check the facts are there.
  • An agent run: ask a browser agent to complete the task, and note where it hesitates or fails.

Whether you need an MCP server as well is covered in whether websites need MCP to be ready for AI agents.

How do you know agents are visiting?

Some agents identify themselves in their user agent strings, and some run in ordinary browsers on users' behalf and are hard to tell apart from people. Server logs, analytics segments and patterns such as rapid, form-focused sessions give partial visibility. How online stores track this is covered in whether online stores can track AI agent interactions with their websites.

What might an illustrative case look like?

Consider an illustrative home goods retailer whose product pages looked polished but used custom components throughout. The "Add to basket" control was a div with an icon, size options were a custom dropdown, and delivery costs appeared only after an address was entered. A browser agent asked to buy a specific lamp chose the wrong size and gave up at delivery. The retailer replaced the custom controls with native elements and accessible names. It labelled every form field and showed standard delivery costs on the product page. Its keyboard-only run, which had failed, now succeeded, as did the agent run. Its Lighthouse accessibility score rose as a side effect.

What do people ask about AI agents and websites?

Is this different from accessibility work?

Largely no. Most fixes that help agents are standard accessibility good practice, which also helps people.

Should we block AI agents?

Block abusive automation, but think before blocking agents acting for real customers. They may be completing a purchase.

Will agents replace visits from people?

Not entirely. They add a new kind of visitor. Sites that work well for keyboard and screen reader users are already most of the way to working for them.

Do we need special files for agents?

Not to start. Clear HTML and accessible controls come first. Proposed standards such as WebMCP are still experimental.

Published by The Kovmere Team · updated Sep 29, 2026 · More resources