Enterprise Log Share: REST API

Cloudflare’s Enterprise customers have access to the Enterprise Log Share service, a RESTful API that allows our customers to consume request logs over HTTP. Our REST API provides an easy method for customers to access a domain’s request logs using their existing client API key.

Some highlights over the previous SFTP-based approach include improved compression (reducing log file sizes by up to 30%); support for querying by a single RayID, a defined time period or from the last RayID downloaded; and a dedicated multi-user role for log access.

These logs contain data surrounding the connecting client, the request’s path through Cloudflare’s network, and the response from the origin, and are extremely useful for enriching existing logs on an origin server. 

How Does It Work?

The service follows the same authentication scheme as the client API. Authentication is described by following the detailed instructions located here - https://api.cloudflare.com/#requests.

In addition, you will also need to retrieve the zone ID (‘zonetag’) for the zone in question from the /zones endpoint: https://api.cloudflare.com/#zone-list-zones

The ELS Schema

The log schema can be found here.

Command Line (CLI) Client

Cloudflare has an open-source command-line client that can be used for interacting with our Enterprise Log Share API directly: https://github.com/cloudflare/logshare

Endpoint Syntax

All endpoints are hosted on https://api.cloudflare.com/

Download a single log derived from the RayID

HTTP Method: GET

Endpoint: /client/v4/zones/:zone_tag/logs/requests/:rayid

Parameter

Description

Type

Required

Notes

zonetag

zone identifier tag

string

yes

32 characters in length

rayid

unique request identifier

string

yes

16 characters in length

 

Download logs starting from a RayID

HTTP Method: GET

Endpoint: /client/v4/zones/:zone_tag/logs/requests?start_id=<rayid>[&end=<unix timestamp>&count=<number>]

Parameter

Description

Type

Required

Notes

zonetag

zone identifier tag

string

yes

32 characters in length

start_id

unique request identifier

string

yes

16 characters in length

end

end time boundary

integer

no

unix timestamp in seconds (UTC)

count

number of records to return before stopping

integer

no

integer value

 

Download logs starting from a specific timestamp

HTTP Method: GET

Endpoint: /client/v4/zones/:zone_tag/logs/requests?start=<Unix timestamp>[&end=<unix timestamp>&count=<number>]

Parameter

Description

Type

Required

Notes

zonetag

zone identifier tag

string

yes

32 characters in length

start

unique request identifier (inclusive)

integer

yes

unix timestamp in seconds (UTC)

end

end time boundary (exclusive)

integer

no

unix timestamp in seconds (UTC)

count

number of records to return before stopping

integer

no

integer value

Example cURL

This will download a one minute interval of logs with GZIP enabled.

curl -sv -o example_file.log.gz -X GET -H "Accept-encoding: gzip" -H "X-Auth-Email: [email protected]" -H "X-Auth-Key: abcdefghijklmnop1234567890" "https://api.cloudflare.com/client/v4/zones/1a2b3c4d5e6f7g8h9i0j1kl2m3n4/logs/requests?start=1446189400&end=1446189460"

Status Codes

  • HTTP 200 (OK) - Issued when logs exist for the given time period.
  • HTTP 204 (No Content) - there were no logs for the given period or log storage not configured for the domain.
  • HTTP 400 (Bad Request) - the API call contained an invalid query parameter (invalid time range or RayID)

 

FAQ:

Q: What format are the logs?

A: The logs are presented in JSON format. Cloudflare provides the schema in the JSON Schema format, which documents the fields/objects in an ELS log entry. The schema itself is attached to this KB article.

Q: When is a log available?

A: The log for a request is typically available within thirty (30) minutes of the request taking place under normal conditions. We deliver logs ordered by the time that the logs were created, i.e. the timestamp of the request when it was received by the edge. Given the order of delivery, we recommend waiting a full thirty minutes to ingest a full set of logs. This will help ensure that any congestion in the log pipeline has passed and a full set of logs can be ingested.

Q: What are ELS logs useful for?

A: Logs can be used to correlate requests to Cloudflare’s against existing origin server logs, track WAF events for specific URIs, and perform top-N analysis for client IPs, countries, or user agents, etc.

Q: How can I become an Enterprise customer and get access to ELS?

A: You can contact our Enterprise team via our web form.

Q: How long are logs retained?

A: 72 Hours

Q: Why isn't a HTTP 206 Partial Content returned when logs aren't up to date?

A: As log updating may be delayed to congestion or other networking issues, we will serve an HTTP 200 OK instead of partial content because these logs are still being aggregated and eventually will be included. 

Still not finding what you need?

The Cloudflare team is here to help. 95% of questions can be answered using the search tool, but if you can’t find what you need, submit a support request.

Powered by Zendesk