78%
of sites in Cloudflare's filtered sample had a robots.txt file. The scan started with 200,000 popular domains.
Cloudflare Agent ReadinessFree website agent readiness white paper
Is Your Website AI Agent Ready?
Learn how to make your website’s information easier for AI agents to find and read, and decide whether agents should be able to take actions on behalf of visitors.
- 10-page guide
- Includes research and links to sources
- Download the PDF after filling out the form
Inside the guide
What you’ll learn
- What agents may try to do on information sites, web apps, and online stores
- Four areas to review: finding information, understanding it, checking trust, and taking action
- A plain-language guide to Cloudflare's Is It Agent Ready checks and which ones matter when
- A scorecard, security checks, and a 30-day plan for improvements
From the research
Research cited in the guide
Here are a few findings discussed in the guide. Follow the source links for the full studies and their context.
3.9%
supported Markdown content negotiation in Cloudflare's initial dataset.
Cloudflare Agent Readiness5 areas
organize the live tester: discovery, content, bot control, protocol discovery, and commerce.
Is Your Site Agent-Ready?Common questions
Before you download
What does it mean for a website to be AI agent ready?
It helps agents find and read information you make public. If you offer tools for agents to use, those tools still need to respect your access rules, the user’s consent, and permissions for individual records.
Does every website need an MCP server or WebMCP?
No. Start with accurate content, well-structured pages, stable links, an up-to-date sitemap, and clear rules for crawlers. Add tools when there’s a specific customer or partner task they can handle more safely than browser automation.
Does a high Cloudflare Agent Readiness score prove the site is ready?
No. It helps you check technical features, but you still need to review content accuracy, accessibility, security, and how the site works for users. It also can’t tell you whether a tool is worth offering to agents.
Want help applying the guide?
Tell us about the task or system you’re working on. We can help you assess the options, plan a trial, or get the project started.