How to Set Up Load Balancing & Intelligent Failover on Cloudflare (using our API)

load-balancing-diagram-overview.png

Introduction

Built on Cloudflare's highly-available and DDoS-resilient Anycast DNS network, our Load Balancing delivers three key features:
  • Load balancing and failover: deliver traffic evenly across healthy servers, automatically failing over when we see them as unhealthy.
  • Active health checks: set up health checks to monitor your servers at configurable intervals, and look for specific status codes, response text, and/or timeouts. We’ll check your servers from each our data-centers, so only the data-centers that can’t reach you have to fail over.
  • Geographic control: direct visitors in Europe to your European datacenter, US visitors to your North American datacenter, or dive-deeper and configure traffic at a regional-level.

Prerequisites

This guide requires that you have:
  • An Enterprise account with Load Balancing enabled, or existing membership in our Early Access program.
  • Familiarity with REST APIs
  • Our Load Balancing API documentation

Terminology

Cloudflare’s Load Balancer has three major components:
 
  • A "Monitor" - the configuration we'll use to determine whether your servers are healthy or unhealthy. This includes whether we check over HTTP or HTTPS, the status code(s) we look for, the interval at which we check, and more.
  • A "Pool" - a group of servers (or endpoints), each identified by their IP address or hostname. You can configure multiple Pools, and configure a failover priority (Pool A -> Pool B -> Pool C) as needed. If you're familiar with DNS terminology, think of a Pool as a record set” - except we only return addresses that are considered healthy.
  • A "Load Balancer", in Cloudflare terms, is a DNS hostname--e.g. www.example.com--that you want traffic to be load-balanced for. A Load Balancer defines which pools it wants to use, in the order they should be used in.
 
Note: you can re-use Monitors and Pools across many Load Balancers. Your .co.uk domain might use a different ordering of Pools (favoring your London servers, for instance) than your .com.au domain.

What We're Creating

We're going to set up a "active-passive" failover setup: we'll send traffic to the servers in our
active Pool until it fails (defined by a threshold we set). Traffic will then failover to the
passive Pool.
 
Here's our key pieces:
  • Hostname: www.example.com
  • Four (4) servers: two primary (app-server-1, app-server-2) and two secondary (backup-server-1, backup-server-2)
  • One location (we’ll tackle adding multiple geographic locations after)
If you're only looking to configure "active-active" failover--where all servers receive
traffic at once: that's even easier. You would just create one Pool that contains all of the
servers/endpoints.
 
Note: Enterprise customers with a multi-user organization should use the /organizations/:org_id/load_balancers/:id  endpoint format instead of /user/load_balancers/:id. The organization ID can be fetched from the organizations API endpoint.

Step 1: Create Your First Monitor

Let’s create our Monitor. A Monitor describes how we will check the status of our servers, and isn’t tied to a particular server. Because of this, we can share this across both of our pools. This also means that changes to our Monitor are automatically reflected across all pools that use it.
 
Here’s our proposed configuration (it should look straightforward):
 
# POST https://api.cloudflare.com/client/v4/user/load_balancers/monitors
{
  "description": "www.example.com Health Check",
  "type": "https",
  "interval": 60,
  "retries": 2,
  "timeout": 5,
  "path": "/health",
  "header": {
    "Host": ["www.example.com"]
  },
  "method": "GET",
  "expected_body": "",
  "expected_codes": "2xx",
  "follow_redirects": false
}
 
Importantly, we define a Host header, so our web-server knows which virtual host’ to serve. In most cases, this will be the same hostname as we intend to create the Load Balancer itself with.
 
Note: Creating a Monitor won’t start any health checks, as you haven’t configured which servers to check yet.

Step 2: Create Your Pools

Let’s create our two Pools. Make sure you have:
 
  • The IP addresses or hostnames of your servers
  • The ID of the Monitor you created above - which you can re-fetch via GET /load_balancers/monitors .
  • An email address (we’ll send health change UP/DOWN emails here)
 
# POST https://api.cloudflare.com/client/v4/user/load_balancers/pools
{
  "name": "primary-dc-1",
  "description": "Primary data center - Provider XYZ",
  "monitor": "f1aba936b94213e5b8dca0c0dbf1f9cc",
  "origins": [
    {
      "name": "app-server-1",
      "address": "0.0.0.0",
    },
    {
      "name": "app-server-2",
      "address": "0.0.2.0",
      "enabled": true
    }
  ],
  "notification_email": "[email protected]"
}
 
If this succeeded, you should get the following response:
 
{
  "result": {
    "id": "17b5962d775c646f3f9725cbc7a53df4",
    "created_on": "2014-01-01T05:20:00.12345Z",
    "modified_on": "2014-01-01T05:20:00.12345Z",
    "description": "Primary data center - Provider XYZ",
    "name": "primary-dc-1",
    "enabled": true,
    "monitor": "f1aba936b94213e5b8dca0c0dbf1f9cc",
    "origins": [
      {
        "name": "app-server-1",
        "address": "0.0.1.0",
        "enabled": true
      },
      {
        "name": "app-server-2",
        "address": "0.0.2.0",
        "enabled": true
      }
    ],
    "notification_email": "[email protected]",
    "healthy": true
  },
  "success": true,
  "errors": [],
  "messages": []
}
 
If you receive an error (or errors) instead, check the error message for a suggestion. If you received a more opaque error, and you’re using curl on the command-line, check that any shell escaping isn’t breaking your JSON request body.
 
Pool #2:
 
# POST https://api.cloudflare.com/client/v4/user/load_balancers/pools
{
  "name": "secondary-dc-1",
  "description": "Secondary data center - Provider QRS",
  "monitor": "f1aba936b94213e5b8dca0c0dbf1f9cc",
  "origins": [
    {
      "name": "backup-server-3",
      "address": "0.0.3.0",
    },
    {
      "name": "backup-server-4",
      "address": "0.0.4.0",
    }
  ],
  "notification_email": "[email protected]"
}
 
You should get a similar response as you did before.
 
As soon as a Pool has been successfully created, we’ll start running health checks against it from each of our data centers.This means that failover is extremely granular, so you may see a reasonable amount of requests from us if you’re running a smaller site or service.
 
Note: We’ll send health status notifications automatically. These emails are triggered when we see your origin servers and/or Pools as unhealthy, and/or when they return to a healthy status.
 
Step 2.1: Check the Health of your Pools
 
If you want to verify that we’re seeing your Pools as healthy, and that we see them as unhealthy when they are indeed down, we provide health” metadata across our Pool endpoints.
 
You’ll see a healthy  field inside each Pool object. We’ll show true/false based on what 70% of our network is currently seeing.
 
# GET /load_balancers/pools *OR* GET /load_balancers/pools/:pool_id
{
  ...
  "healthy": true
  ...
}
 
You can also get a detailed, per point-of-presence breakdown of the health of a Pool from the pools/:pool_id/health endpoint:
 
{
  "result": {
    "pool_id": "ff02c959d17f7bb2b1184a202e3c0af7",
    "pop_health": {
      "Newark, NJ": {
        "healthy": true,
        "origins": [
          {
            "52.22.68.195": {
              "healthy": true,
              "rtt": "161.1ms",
              "failure_reason": "No failures",
              "response_code": 200
            }
          },
          # Each origin will appear in this array
        ]
      }
    },
    # Each Cloudflare Point-of-Presence will be listed here.
  },
  "success": true,
  "errors": [],
  "messages": []
}
 
This endpoint can be verbose, but can be useful if you need to dive down into to more isolated failures in a specific region. For most use-cases, the global health returned from the /load_balancers/pools  endpoint will be more useful.

Step 3: Create the Load Balancer

Thus far we’ve:
  • Created a Monitor
  • Created two (2) Pools with our origin servers
  • Reviewed how to check the health of our Pools (via email notifications and the API)
In order to start delivering traffic to our Pools, we need to attach them to a DNS hostname, and define the failover priority of those Pools.
  • default_pools defines the failover ordering. We’ll steer traffic to the first pool in the list, and failover to the next healthy Pool, in order.
  • region_pools is where we would map Pools to specific regions (we’ll detail how that works further down).
  • fallback_pool is the pool of last resort”: if all other Pools are unhealthy, we’ll send traffic here. For most users, this Pool can be your secondary/passive Pool.
  • name is the public DNS hostname to use for your Load Balancer.
Note: If you have an existing DNS record of the same name as your Load Balancer, the Load Balancer will take precedence. The DNS record will not be used unless you delete the Load Balancer object.
 
Before you create the Load Balancer object, you’ll need to get the Cloudflare zone ID for that zone here: https://api.cloudflare.com/#zone-list-zones
 
With the ID in hand, you can create your Load Balancer. Within a few seconds (at most!) of this request, we’ll start steering traffic to the configured Pools.
 
# POST /zones/:zone_id/load_balancers
{
  "description": "Load Balancer for www.example.com",
  "name": "www.example.com",
  "ttl": 30,
  "proxied": true,
  "fallback_pool": "ff02c959d17f7bb2b1184a202e3c0af7",
  "default_pools": ["17b5962d775c646f3f9725cbc7a53df4", "ff02c959d17f7bb2b1184a202e3c0af7"]
  "region_pools": {}
}
 
You should receive a success from our API. 

(Optional) Step 4: Geographic (Regional) Failover

If you have servers in different geographic regions, you may want to steer traffic to Pools in a different order depending on where those users are connecting from. For example, your European visitors should land on your European pool first, and then only your US pool if the European pool is down. North American users would have the reverse configuration.
 
Our plan:
  • Direct EU users to our secondary Pool first (if it were located in Europe) and our primary pool second (if it were located in North America).
  • Direct North American users to our US pool first, and EU pool second.
  • All other regions can just use our default_pools . If we wanted to prescribe a specific order for every region (refer to the API reference for a list of regions) then we would need to configure a failover priority for each region code.
Note that to edit an existing object we use PUT - replacing it with the intended final” configuration.
 
# PUT /zones/:zone_id/load_balancers
{
  "description": "Load Balancer for www.example.com",
  "name": "www.example.com",
  "ttl": 30,
  "proxied": true,
  "fallback_pool": "ff02c959d17f7bb2b1184a202e3c0af7",
  "default_pools": ["17b5962d775c646f3f9725cbc7a53df4", "ff02c959d17f7bb2b1184a202e3c0af7"]
  "region_pools": {
  "WNAM": ["17b5962d775c646f3f9725cbc7a53df4", "ff02c959d17f7bb2b1184a202e3c0af7"],
  "ENAM": ["17b5962d775c646f3f9725cbc7a53df4", "ff02c959d17f7bb2b1184a202e3c0af7"],
  "EU": ["ff02c959d17f7bb2b1184a202e3c0af7", "17b5962d775c646f3f9725cbc7a53df4"]}
}
 
Our view of North America is split into two. If you only define WNAM”, then east-coast traffic will use the default_pools  configuration by default, as will other undefined’ regions.
 
To test this, you’ll need to use a client in those locations: a DigitalOcean droplet or VPN service can come in useful if you don’t have something set up already.

Wrap:

So what’s next?
  • Refer to the API reference: we document all of the optional fields and methods (GET, POST, PUT, DELETE) for each endpoint.
  • If you have any questions or need some help, reply to your Early Access ticket and we’ll be happy to answer.
You can also refer to our knowledge-base articles on Load Balancing: https://support.cloudflare.com/hc/en-us/sections/206719048-Load-Balancing
Still not finding what you need?

The CloudFlare team is here to help. 95% of questions can be answered using the search tool, but if you can’t find what you need, submit a support request.

Powered by Zendesk