Does Cloudflare crawl my site?

The only feature that Cloudflare uses that creates a crawl of your site is Always Online so we can display a limited cache of your site if your server goes down. The useragent we use is:

Mozilla/5.0 (compatible; Cloudflare-AlwaysOnline/1.0; +http://www.cloudflare.com/always-online

We crawl sites based on the Cloudflare pricing plan for the domain.

Free customers once every 7 days
Pro customers once every 3 days
Business and Enterprise customers daily.

Still not finding what you need?

The CloudFlare team is here to help. 95% of questions can be answered using the search tool, but if you can’t find what you need, submit a support request.

Powered by Zendesk