Does Cloudflare crawl my site?

The only feature that Cloudflare uses that creates a crawl of your site is Always Online so we can display a limited cache of your site if your server goes down. The useragent we use is:

Mozilla/5.0 (compatible; Cloudflare-AlwaysOnline/1.0; +

We crawl sites based on the Cloudflare pricing plan for the domain.

Free customers once every 7 days
Pro customers once every 3 days
Business and Enterprise customers daily.

Not finding what you need?

95% of questions can be answered using the search tool. This is the quickest way to get a response.

Powered by Zendesk