Learn how Always Online keeps a limited version of your site online if your origin web server goes offline. Cloudflare Always Online with Internet Archive integration is currently available as a Cloudflare Beta feature.
Overview
Cloudflare’s Always Online feature is now integrated with the Internet Archive so that visitors can access a portion of your website even when your origin server is unreachable and a Cloudflare-cached version is not available.
When your origin is unreachable, Always Online checks Cloudflare’s cache for a stale or expired version of your website. When none exists, Cloudflare will go to the Internet Archive to fetch and serve static portions of your website.
When you enable Always Online with Internet Archive integration, Cloudflare shares your hostname and popular URL paths with the archive so that the Internet Archive’s crawler stores the pages you want archived. When submitting targets to the crawler, Cloudflare identifies the most popular URLs found among GET requests that returned a 200 HTTP status code in the previous 5 hours.
Note that Cloudflare does not save a copy of every page of your website, and it cannot serve dynamic content while your origin is offline. If the requested page is not in Cloudflare’s Always Online cache or the Internet Archive's Wayback Machine, the visitor sees the actual error page caused by the offline origin web server.
A Cloudflare crawler identifies resources to place in the Always Online cache. When Internet Archive integration is enabled, Cloudflare tells the Internet Archive what pages to crawl and how often. To ensure stability of service, Cloudflare limits the crawling interval. Limits vary by Cloudflare plan.
- Free customers once every 30 days.
- Pro customers once ever 15 days.
- Business and Enterprise customers once every 5 days.
Visitor experience
When Always Online with Internet Archive integration is enabled, visitors see a banner at the top of the web page notifying them they are visiting an archived version of the website. Visitors can click the Refresh button to check whether the origin has recovered and fresh content is available.
When a visitor requests content for an offline website, Cloudflare returns an HTTP response status code in the range 520–527, depending on the issue. These status codes indicate that the origin is unreachable.
When the requested page is not in the Always Online cache and Internet Archive integration is enabled, Cloudflare checks the archive and serves the most recently archived version of the page.
Visitors who interact with dynamic parts of a website, such as a shopping cart or comment box, will see an error page caused by the offline origin web server.
Enable Internet Archive integration
Always Online with Internet Archive integration is a Beta feature and is not enabled by default. To enable the integration, follow these steps:
- Log in to your Cloudflare account.
- Choose the domain for which you want to enable Always Online with Internet Archive integration. The Cloudflare dashboard displays.
- Click the Caching app.
- In the Caching app, select the Configuration tab.
- To enable Always Online, scroll to the Always Online Beta card and toggle it to On.
- To enable Internet Archive integration, click Update.
To use Cloudflare's Always Online cache without Internet Archive integration, switch the toggle to On and do not update.
Alternatives to globally enabling Always Online include:
- Using Cloudflare Page Rules to enable Always Online, or
- Allowing your origin web server to determine which content to cache for display if your origin web server is offline:
- Disable Always Online,
- Set Origin Cache Control for your resources, and
- Enable stale-if-error at your origin.
Best practices and limitations
When enabling Always Online with Internet Archive integration, observe these best practices:
- Allow requests from Cloudflare IP addresses. Origin servers for domains proxied through Cloudflare receive requests from Cloudflare IP addresses. To avoid blocking these requests, follow the guidelines for allowing Cloudflare IP addresses. When you observe the Always Online banner while your origin web server is online, your origin web server or hosting provider are likely blocking or rate limiting Cloudflare requests.
- Configure your origin server’s Cache Control header. To ensure Always Online caches resources for your site, do not set your origin server’s Cache Control header to no-cache, must-revalidate, or max-age=0.
- Consider potential conflicts with Cloudflare features that transform URIs. Always Online Beta with Internet Archive integration may cause issues with Page Rules and other Cloudflare features that transform URIs, due to the way the Internet Archive crawls pages to archive. Specifically, some redirects that take place at the edge may cause the Internet Archive's crawler not to archive the target URL. Before enabling Origin Cache Control, review how Cloudflare caches resources by default as well as any Page Rules you have configured so that you can avoid these issues. If you experience problems, disable Always Online Beta.
Do not use Always Online with:
- Custom Hostnames (SSL for SaaS),
- API traffic,
- An IP Access Rule or Firewall Rule that blocks the United States, or
- A Cache Everything Page Rule that configures an Edge Cache TTL lower than the Always Online crawl frequency pertaining to your domain plan type.
There are limitations with the Always Online functionality:
1. Always Online is not immediately active for sites recently added to Cloudflare due to:
- DNS record propagation (can take 24-72 hours), or
- Always Online has not initially crawled the website.
2. Cloudflare cannot show private content behind logins or handle form submission (POSTs) if your origin web server is offline.