On 15 September 2026 Cloudflare changed how websites control AI crawlers. The old Block AI Bots switch was a single choice, and it often treated search and training as the same thing. Cloudflare replaced it with separate Search, Training and Agent controls, and added a setting called Disallow AI Training.
The practical result is the split most businesses want. You can stay discoverable in traditional search, and refuse to let that same crawl train AI models.
What the three controls do
- Search covers crawlers that index your site for traditional search results, including Google, Bing and Apple.
- Training covers crawlers that use your content to train AI models.
- Agent covers crawlers that browse on behalf of an AI assistant.
Disallow AI Training is a Training setting. Cloudflare publishes that preference into your robots.txt through Bot Preference Sync. Mixed-use crawlers that both search and train can keep crawling for search when their operator honours the split. Training-only crawlers from other operators can be blocked at the edge, so they never fetch the page.
Cloudflare labels Applebot, Googlebot and Bingbot as accountable. Apple, Google and Microsoft honour the split, or have committed to honour it. That is why you can leave Search allowed and still disallow training, and remain in Google, Bing and Apple search.
The setting is available on all Cloudflare plans, including Free.
What auto-migration usually did
Existing Block AI customers were migrated automatically. The typical result is Search set to Allowed, and Training set to Disallow AI Training. Typical is not the same as checked. A site that meant to stay hidden from search, or a site whose WordPress or SEO plugin already writes its own robots.txt, can end up with a combination nobody chose on purpose.
Setting Search to Block is a different decision. That can stop Googlebot, Bingbot and Applebot from crawling you, which is how a site drops out of traditional results. Leave Search on Allowed unless hiding from search is what you mean to do.
Training is separate from AI Overviews
Cloudflare's Training control decides whether crawlers may use your pages to train models. It is separate from the generative AI report in Google Search Console, and it does not decide whether Google shows an AI summary above normal results. Training is not the same thing as an AI Overview.
SEO and GEO are separate services as well. SEO is how you rank in ordinary results. GEO is how your business appears in AI answers, and it is not part of an SEO plan. This Cloudflare change is a hosting and CDN check: who may crawl the site, and for what purpose.
What to do
- Open Cloudflare. Go to Security, then the AI crawl or bot settings.
- Confirm Search is still Allowed unless you mean to hide the site from traditional search.
- Decide Training from your content strategy. Disallow AI Training refuses training and leaves accountable search crawlers in place. Block also stops those crawlers, including for search.
- Read the live robots.txt. Check that WordPress, an SEO plugin, or another tool is not writing rules that conflict with Bot Preference Sync.
If Ven manages your hosting or CDN, ask us to confirm the settings. If you manage Cloudflare yourself, this is about a ten-minute check.
From Ven
Want a hosting and CDN check?
If we look after your hosting or Cloudflare, we can confirm Search is still allowed and Training matches what you want.
The bottom line
You can stay in Google, Bing and Apple search and still refuse AI training. You do need to look, because Cloudflare already moved many existing Block AI setups across, and a plugin can still disagree with the new robots.txt.
Related reading: why your business can't afford free web hosting.
Also this week: the AI Max auto-upgrade for Search campaigns and Google's September 2026 spam update.