Using cURL when Troubleshooting with Cloudflare

Overview

cURL is a command line tool that is used to send or receive HTTP/HTTPS requests using URL syntax.

For CSUP, this tool is useful for debugging:

  • HTTP/HTTPS Performance
  • HTTP Error Responses 
  • HTTP Headers
  • APIs
  • Comparing Server/Proxy Responses
  • SSL Certificates

Installing cURL (Windows)

If the local system being used for testing/troubleshooting is not a Linux/UNIX based system, then cURL will likely not be installed by default for use on the command line in Windows.

The open source community at curl.haxx.se provides an install wizard that can be used for Windows users. Please follow the following instructions if using this method for installing cURL:

  1. Select Type of Package: curl executable
  2. Select Operating System: Windows / Win32 or Win64
  3. Select for What Flavour: Generic
  4. Select which Win32 Version (only if you selected Windows / Win32 in step 2): Unspecified

If using x64 based systems, please refer to the 64-bit executables from curl.haxx.se.

General Use

A simple command to send a standard HTTP GET request would be:

curl -svo /dev/null http://example.com/

This command will return an output detailing the HTTP response and request headers, while dumping the page body to the local null device (which discards any information written to it immediately). 

Trying 104.16.39.188...
* Connected to whiskytango.us (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: whiskytango.us
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sat, 09 Jul 2016 11:53:37 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: __cfduid=d1f3639ea02e3abff666d; expires=Sun, 09-Jul-17 11:53:37 GMT; path=/; domain=.whiskytango.us; HttpOnly
< Cf-Railgun: direct (waiting for pending WAN connection)
< Last-Modified: Tue, 18 Aug 2015 09:05:34 GMT
< Server: cloudflare-nginx
< CF-RAY: 2bfb939a0a57440e-SFO-DOG
<
{ [2291 bytes data]
* Connection #0 to host whiskytango.us left intact]]>

The above output is generally helpful to confirm the HTTP response, and whether Cloudflare is currently proxying traffic for the site or not (evident in this case by the Server and CF-RAY headers).

Manipulating cURL Outputs

In some scenarios, you may only need to review a specific header value or bit of information, instead of the entire cURL output/response.

In these cases, cURL can be combined with commands like grep/egrep and manipulate the stdout (standard output) with 2>&1:

curl -svo /dev/null http://whiskytango.us/img/sg50/IMG_0190.jpg 2>&1 | grep "CF-"
< CF-Cache-Status: HIT
< CF-RAY: 2c07be0acb1d440e-SFO-DOG

In the example above, this cURL was used to parse out the CF-Cache-Status and CF-RAY headers (helpful method for reviewing/troubleshooting caching).

If multiple variables need to be specified for parsing out from the standard cURL output, egrep can be used:

curl -svo /dev/null http://whiskytango.us/img/sg50/IMG_0190.jpg 2>&1 | egrep "Date|CF-|HTTP/"
> GET /img/sg50/IMG_0190.jpg HTTP/1.1
< HTTP/1.1 200 OK
< Date: Sun, 10 Jul 2016 23:35:54 GMT
< CF-Cache-Status: HIT
< CF-RAY: 2c07d5b6bed7440e-SFO-DOG'

2>&1 is a construct that indicates stderr (standard error) should be redirected (and thus included) in the stdout (standard output). This is helpful for providing more verbose information, and being able to parse the cURL output into another command (like grep).

Commonly Used Flags

-svo /dev/null

This series of flags specifies a verbose response, while dumping the output/body of the page to the local /dev/null item, which discards that information immediately and is not saved. This is used to provide a clean display of information, such as the request/response headers. Here is a breakdown of these options:

-s: Silent mode. Removes show progress meter or error messages.

-v: Enables verbose output. 

-o /dev/null: sets output to a file. In this case we are sending the page body to the /dev/null item. 

--header "HEADER: VALUE"

The --header option above is useful for sending specific request headers, such as Host, Set-Cookie, or even a custom header that is used in a unique customer application/platform. 

--user-agent "USERAGENTSTRING"

In some cases, web applications/servers will have rules in place to block the default cURL user-agent string, so it is helpful to send a legitimate user agent string from a current/known browser. There may also be cases where tests against a specific UA is needed. If needed, a database of User-agent strings can be found here.

--resolve hostname:port:DESTINATIONIPADDRESS http(s)://hostname/URL]]>

The resolve option sets a custom address for a specific host and port. This provides the ability to send curl requests using a specified address and prevent the otherwise normally resolved address to be used (command line alternative to modifying the local /etc/hosts file). This option is extremely useful for comparing server responses from the origin server and against requests being proxied by Cloudflare. 

Debugging HTTP Errors

When troubleshooting HTTP Errors, try cURLing against the origin to confirm the response without being proxied by Cloudflare.

This can be achieved two ways, either by specifying the Host header and cURLing against the site's source IP address, or by utilizing the --resolve option. 

curl -svo /dev/null --header "Host: example.com" http://123.123.123.123/
curl -svo /dev/null --resolve example.com:80:123.123.123.123 http://example.com/

Here is an example where a 520 is being seen for requests being proxied:

curl -svo /dev/null http://whiskytango.us/
*   Trying 104.16.40.188...
* Connected to whiskytango.us (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: whiskytango.us
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 520 Unknown Origin-Error
< Date: Sun, 10 Jul 2016 19:55:33 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: __cfduid=d5a459e3960ee3254; expires=Mon, 10-Jul-17 19:55:18 GMT; path=/; domain=.whiskytango.us; HttpOnly
< Pragma: no-cache
< X-Frame-Options: SAMEORIGIN
< Server: cloudflare-nginx
< CF-RAY: 2c069292c8e3440e-SFO-DOG
<
{ [3685 bytes data]
* Connection #0 to host whiskytango.us left intact]]>

Running a cURL to the origin is now showing that empty replies are being sent, which would confirm the root cause of the 520 error:

curl -svo /dev/null --resolve whiskytango.us:80:123.123.123.123 http://whiskytango.us/
* Added whiskytango.us:80:123.123.123.123 to DNS cache
* Hostname whiskytango.us was found in DNS cache
*   Trying 123.123.123.123...
* Connected to whiskytango.us (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: whiskytango.us
> User-Agent: curl/7.43.0
> Accept: */*
>
* Empty reply from server
* Connection #0 to host whiskytango.us left intact]]>

Performance Troubleshooting

Using cURL can be helpful to measure latency or performance degradation for HTTP/HTTPS requests. A quick and easy method is to use both the cURL and time commands:

time curl -svo /dev/null http://example.com/

This shows the total time taken for the request to be completed:

* Trying 104.16.43.188...
* Connected to whiskytango.us (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: whiskytango.us
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 10 Jul 2016 19:10:28 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: __cfduid=de980d61f2dc09; expires=Mon, 10-Jul-17 19:10:28 GMT; path=/; domain=.whiskytango.us; HttpOnly
< Cf-Railgun: direct (waiting for pending WAN connection)
< Last-Modified: Tue, 18 Aug 2015 09:05:34 GMT
< Server: cloudflare-nginx
< CF-RAY: 2c0650e4f76c4408-SFO-DOG
<
{ [925 bytes data]
* Connection #0 to host whiskytango.us left intact
real	0m0.079s
user	0m0.007s
sys	0m0.006s]]>

A more thorough and complete method of performance testing would be to use the -w or --write-out options, which allows the specification of variables for benchmarking that cURL will print in the output. A full list of the variables for this option can be found by running man curl on the command line, or by visiting the manual at curl.haxx.se.

Here is an example cURL run that is measuring several performance vectors in the request transaction:

curl -svo /dev/null https://example.com/ -w "\nContent Type: %{content_type} \
\nHTTP Code: %{http_code} \
\nHTTP Connect:%{http_connect} \
\nNumber Connects: %{num_connects} \
\nNumber Redirects: %{num_redirects} \
\nRedirect URL: %{redirect_url} \
\nSize Download: %{size_download} \
\nSize Upload: %{size_upload} \
\nSSL Verify: %{ssl_verify_result} \
\nTime Handshake: %{time_appconnect} \
\nTime Connect: %{time_connect} \
\nName Lookup Time: %{time_namelookup} \
\nTime Pretransfer: %{time_pretransfer} \
\nTime Redirect: %{time_redirect} \
\nTime Start Transfer: %{time_starttransfer} \
\nTime Total: %{time_total} \
\nEffective URL: %{url_effective}\n" 2>&1

Following the cURL output, the variable values will be displayed. Here is an example from the cURL above:

 > GET / HTTP/1.1
> Host: whiskytango.us
> User-Agent: curl/7.43.0
> Accept: */*
>
{ [5 bytes data]
< HTTP/1.1 200 OK
< Date: Sun, 10 Jul 2016 19:25:06 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: __cfduid=dcfcff083c6e553dc6750134395b8bf711468178706; expires=Mon, 10-Jul-17 19:25:06 GMT; path=/; domain=.whiskytango.us; HttpOnly
< Cf-Railgun: direct (starting new WAN connection)
< Last-Modified: Tue, 18 Aug 2015 09:05:34 GMT
< Server: cloudflare-nginx
< CF-RAY: 2c0666564631440e-SFO-DOG
<
{ [935 bytes data]
* Connection #0 to host whiskytango.us left intact
Content Type: text/html
HTTP Code: 200
HTTP Connect:000
Number Connects: 1
Number Redirects: 0
Redirect URL:
Size Download: 14166
Size Upload: 0
SSL Verify: 0
Time Handshake: 0.076
Time Connect: 0.018
Name Lookup Time: 0.002
Time Pretransfer: 0.076
Time Redirect: 0.000
Time Start Transfer: 0.111
Time Total: 0.129
Effective URL: https://whiskytango.us/

From the output, helpful performance information that can impact load times can be viewed, such as TLS handshake, DNS lookup time, redirects, transfers, upload/download size, etc. 

When using -w or --write-out, a cleaner output of the results can be achieved by denoting a new line with \n before each variable. This will print each variable on a new line, instead of displaying all the metrics in a single line.

Caching

cURL is also helpful for reviewing caching behaviors when reviewing the HTTP response headers. The following HTTP headers should almost always be taken into account when troubleshooting caching with Cloudflare:

CF-Cache-Status

Cache-control/Pragma

Expires

Last-Modified

S-Maxage

Specifics on Cloudflare's caching behavior for can be found in the following KB Article:How do I tell Cloudflare What to Cache?

When using cURL, utilizing the same methods as described in the Debugging HTTP Errors section of this article are helpful to compare caching response headers from the origin, and when requests are being proxied by Cloudflare.

Here is an example that utilizes output manipulation to show that the site's origin is serving three different versions of the same file (noting the Last-Modified header), and confirming why stale cache would have been served intermittently for requests:

curl -vso /dev/null -H "Host: example.com" http://162.242.199.20/courses 2>&1 | egrep "< Date|< Last-Modified|< ETag"
< Last-Modified: Sun, 23 Mar 2014 01:07:13 GMT
< ETag: "4147-4f53bbd33b240"
< Date: Thu, 03 Apr 2014 22:52:58 GMT

curl -vso /dev/null -H "Host: example.com" http://162.242.199.20/courses 2>&1 | egrep "< Date|< Last-Modified|< ETag"
< Last-Modified: Sun, 23 Mar 2014 01:07:26 GMT
< ETag: "4149-4f53bbdfa0f80"
< Date: Thu, 03 Apr 2014 22:53:00 GMT

curl -vso /dev/null -H "Host: example.com" http://162.242.199.20/courses 2>&1 | egrep "< Date|< Last-Modified|< ETag"
< Last-Modified: Sun, 23 Mar 2014 01:03:02 GMT
< ETag: "413d-4f53bae3dbd80"
< Date: Thu, 03 Apr 2014 22:53:43 GMT

Debugging SSL/TLS

Reviewing Certifcates with cURL

Aside from using openSSL for reviewing a site's certificate information and troubleshooting, cURL is another tool that can be used to review the certificate served in the HTTPS request:

curl -svo /dev/null https://whiskytango.us/ 2>&1 | egrep -v "^{.*$|^}.*$|^\* http.*$"

With the command above, the results will display helpful information, such as the TLS handshake and certificate information that was returned for the request (in this case, the proxied request would be served a shared certificate issued by Comodo):

*  Trying 104.16.43.188...
* Connected to whiskytango.us (104.16.43.188) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /usr/local/etc/openssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
* 	 subject: OU=Domain Control Validated; OU=PositiveSSL Multi-Domain; CN=ssl329925.cloudflaressl.com
* 	 start date: 2016-01-04 00:00:00 GMT
* 	 expire date: 2016-12-31 23:59:59 GMT
* 	 subjectAltName: whiskytango.us matched
* 	 issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO ECC Domain Validation Secure Server CA 2
* 	 SSL certificate verify ok.
> GET / HTTP/1.1
> Host: whiskytango.us
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 10 Jul 2016 21:41:04 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: __cfduid=dde6e686bb3fffe36d57863a78828beb11468186864; expires=Mon, 10-Jul-17 21:41:04 GMT; path=/; domain=.whiskytango.us; HttpOnly
< Cf-Railgun: direct (starting new WAN connection)
< Last-Modified: Tue, 18 Aug 2015 09:05:34 GMT
< Server: cloudflare-nginx
< CF-RAY: 2c072d7e8be6440e-SFO-DOG
<
* Connection #0 to host whiskytango.us left intact

The string, 2>&1 | egrep -v "^{.*$|^}.*$|^\* http.*$" can be used to clean up and parse cURL outputs to provide a less noisy display of the TLS handshake/certificate information.

Reviewing the origin certificate (assuming one is installed) can be done by using the --resolve flag:

curl -svo /dev/null --resolve www.example.com:443:ORIGINIPHERE https://www.example.com/

Testing TLS Versions

Testing may need to be done against a specific TLS version (in the event of troubleshooting against legacy browsers, or confirming what the site's origin supports for TLS). 

Here are the options for sending a TLS request using a specific version to negotiate with:

--tlsv1.0

--tlsv1.1

--tlsv1.2

Here is an example cURL sending an HTTPS request using the TLS protocol, version 1.2:

curl -svo /dev/null --tlsv1.1 https://whiskytango.us/ 2>&1 | egrep -v "^{.*$|^}.*$|^\* http.*$"
*   Trying 104.16.40.188...
* Connected to whiskytango.us (104.16.40.188) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /usr/local/etc/openssl/cert.pem
  CApath: none
* TLSv1.1 (OUT), TLS header, Certificate Status (22):
* TLSv1.1 (OUT), TLS handshake, Client hello (1):
* TLSv1.1 (IN), TLS handshake, Server hello (2):
* TLSv1.1 (IN), TLS handshake, Certificate (11):
* TLSv1.1 (IN), TLS handshake, Server key exchange (12):
* TLSv1.1 (IN), TLS handshake, Server finished (14):
* TLSv1.1 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.1 (OUT), TLS change cipher, Client hello (1):
* TLSv1.1 (OUT), TLS handshake, Finished (20):
* TLSv1.1 (IN), TLS change cipher, Client hello (1):
* TLSv1.1 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.1 / ECDHE-RSA-AES128-SHA
* ALPN, server accepted to use http/1.1
* Server certificate:
* 	 subject: OU=Domain Control Validated; OU=PositiveSSL Multi-Domain; CN=ssl329924.cloudflaressl.com
* 	 start date: 2016-01-03 00:00:00 GMT
* 	 expire date: 2016-12-31 23:59:59 GMT
* 	 subjectAltName: whiskytango.us matched
* 	 issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO RSA Domain Validation Secure Server CA 2
* 	 SSL certificate verify ok.
> GET / HTTP/1.1
> Host: whiskytango.us
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Mon, 18 Jul 2016 20:48:41 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: __cfduid=dfdfe3bd6aec2927ab36d3748cd6a30a31468874921; expires=Tue, 18-Jul-17 20:48:41 GMT; path=/; domain=.whiskytango.us; HttpOnly
< Cf-Railgun: direct (starting new WAN connection)
< Last-Modified: Tue, 18 Aug 2015 09:05:34 GMT
< Server: cloudflare-nginx
< CF-RAY: 2c48cbc21031440e-SFO-DOG
<
* Connection #0 to host whiskytango.us left intact

References

curl.haxx.se

Still not finding what you need?

The CloudFlare team is here to help. 95% of questions can be answered using the search tool, but if you can’t find what you need, submit a support request.

Powered by Zendesk