Note: A new endpoint with a more concise format & improved determinism is now available in "Early Access" status: find out more.
Cloudflare’s Enterprise customers have access to the Enterprise Log Share service, a RESTful API that allows our customers to consume request logs over HTTP. Our REST API provides an easy method for customers to access a domain’s request logs using their existing client API key.
Some highlights over the previous SFTP-based approach include improved compression (reducing log file sizes by up to 30%); support for querying by a single RayID, a defined time period or from the last RayID downloaded; and a dedicated multi-user role for log access.
These logs contain data surrounding the connecting client, the request’s path through Cloudflare’s network, and the response from the origin, and are extremely useful for enriching existing logs on an origin server.
How Does It Work?
The service follows the same authentication scheme as the client API. Authentication is described by following the detailed instructions located here - https://api.cloudflare.com/#requests.
In addition, you will also need to retrieve the zone ID (‘zonetag’) for the zone in question from the /zones endpoint: https://api.cloudflare.com/#zone-list-zones
The ELS Schema
The log schema can be found here.
Command Line (CLI) Client
Cloudflare has an open-source command-line client that can be used for interacting with our Enterprise Log Share API directly: https://github.com/cloudflare/logshare
Endpoint Syntax
All endpoints are hosted on https://api.cloudflare.com/
Available endpoints
- /requests: exposes data aggregated by time of request.
- /received (Early Access): exposes data aggregated by time received (the time the event was written to disk in our log aggregation system. Ordering by log aggregation time instead of log generation time results in lower (faster) log share pipeline latency and deterministic/idempotent log pulls. More information here.
Download a single log derived from the RayID
HTTP Method: GET
Endpoint: /client/v4/zones/:zone_tag/logs/requests/:rayid
|
Parameter |
Description |
Type |
Required |
Notes |
|
zonetag |
zone identifier tag |
string |
yes |
32 characters in length |
|
rayid |
unique request identifier |
string |
yes |
16 characters in length |
Download logs starting from a RayID
HTTP Method: GET
Endpoint: /client/v4/zones/:zone_tag/logs/requests?start_id=<rayid>[&end=<unix timestamp>&count=<number>]
|
Parameter |
Description |
Type |
Required |
Notes |
|
zonetag |
zone identifier tag |
string |
yes |
32 characters in length |
|
start_id |
unique request identifier |
string |
yes |
16 characters in length |
|
end |
end time boundary |
integer |
no |
unix timestamp in seconds (UTC) |
|
count |
number of records to return before stopping |
integer |
no |
integer value |
Download logs starting from a specific timestamp
HTTP Method: GET
Endpoint: /client/v4/zones/:zone_tag/logs/requests?start=<Unix timestamp>[&end=<unix timestamp>&count=<number>]
|
Parameter |
Description |
Type |
Required |
Notes |
|
zonetag |
zone identifier tag |
string |
yes |
32 characters in length |
|
start |
unique request identifier (inclusive) |
integer |
yes |
unix timestamp in seconds (UTC) |
|
end |
end time boundary (exclusive) |
integer |
no |
unix timestamp in seconds (UTC) |
|
count |
number of records to return before stopping |
integer |
no |
integer value |
Example cURL
This will download a one minute interval of logs with GZIP enabled.
curl -sv -o example_file.log.gz -X GET -H "Accept-encoding: gzip" -H "X-Auth-Email: [email protected]" -H "X-Auth-Key: abcdefghijklmnop1234567890" "https://api.cloudflare.com/client/v4/zones/1a2b3c4d5e6f7g8h9i0j1kl2m3n4/logs/requests?start=1446189400&end=1446189460"
Status Codes
- HTTP 200 (OK) - Issued when logs exist for the given time period.
- HTTP 204 (No Content) - there were no logs for the given period or log storage not configured for the domain.
- HTTP 400 (Bad Request) - the API call contained an invalid query parameter (invalid time range or RayID)
FAQ:
Q: What format are the logs?
A: The logs are presented in JSON format. Cloudflare provides the schema in the JSON Schema format, which documents the fields/objects in an ELS log entry. The schema itself is attached to this KB article.
Q: When is a log available?
A: The log for a request is typically available within thirty (30) minutes of the request taking place under normal conditions. We deliver logs ordered by the time that the logs were created, i.e. the timestamp of the request when it was received by the edge. Given the order of delivery, we recommend waiting a full thirty minutes to ingest a full set of logs. This will help ensure that any congestion in the log pipeline has passed and a full set of logs can be ingested.
Q: What are ELS logs useful for?
A: Logs can be used to correlate requests to Cloudflare’s against existing origin server logs, track WAF events for specific URIs, and perform top-N analysis for client IPs, countries, or user agents, etc.
Q: How can I become an Enterprise customer and get access to ELS?
A: You can contact our Enterprise team via our web form.
Q: How long are logs retained?
A: 72 Hours
Q: Why isn't a HTTP 206 Partial Content returned when logs aren't up to date?
A: As log updating may be delayed to congestion or other networking issues, we will serve an HTTP 200 OK instead of partial content because these logs are still being aggregated and eventually will be included.