Identify, monitor, and mitigate automated requests with Bot Management, a Cloudflare mitigation solution based on machine learning. It complements our suite of firewall products including Web Application Firewall (WAF), Rate Limiting, and IP Reputation.
- What is Cloudflare Bot Management?
- What is different between Bot Management and the bot protection that Cloudflare already offers?
- Where do I activate Bot management in the Cloudflare dashboard?
- I enabled Bot Management in Log/Simulate mode. What should I look for?
- How does Machine Learning work?
- What is the difference between the threat score and bot score?
- What is cf.client.bot?
- I'm a good bot and want to be added to the white list (cf.bot_management.verified_bot). What should I do?
What is Cloudflare Bot Management?
By leveraging machine learning across the 13 million Internet properties proxied by Cloudflare, we are able to assign a score to every request. Cloudflare creates dynamic Firewall Rules to match against malicious bot requests whenever the request score falls below a specific threshold.
Key Cloudflare Bot Management differentiators include:
- More accurate bot identification based on a large training set: Our curated training set comprises millions of requests made against 13M Internet properties.
- Reduced latency and faster performance thanks to decisions made at the edge: Since such a large number of requests traveling through the Cloudflare network get their score updated, we push the machine learning results to our edge. Then, requests can be evaluated at the edge instead of the origin. This reduces latency and prevents uncached requests from impacting origin server CPU.
Cloudflare Bot Management is part of a consolidated solution. Many different attack vectors can lead to compromised protection. Denial of service attacks, poor access controls, and SQL injection can aggravate bot-based attacks. To guard against these, security teams can design comprehensive protection with less training and context switching using Cloudflare's consolidated solution.
What is different between Bot Management and the bot protection that Cloudflare already offers?
Cloudflare Bot Management focuses on explicit bot mitigation as compared to our current solution revolving around the WAF and rate limiting.
Bot Management currently covers seven major use cases: Credential and credit card stuffing, content scraping, and other types of deception including, spam, registration, marketing, and ad-click fraud. Bot management is designed to detect and block bad bots based on the following three mitigation methods:
Machine learning (ML)
By applying ML across 13M Internet properties, Cloudflare creates a reliable bot score that can be used to create rules for blocking requests based on the likelihood that they come from a bot.
Behavioral analysis (BA)
Behavioral analytics detect and block abnormal requests based on HTTP sessions. This approach is user-agent agnostic and identifies potential bots based on actual metrics.
You can combine or use the above methods separately.
The biggest advantages of Bot Management over the standard solution are:
- More reliable bot identification based on a large training set
- Faster performance with decisions made closer to the client
Where do I activate Bot management in the Cloudflare dashboard?
Once enabled for your Enterprise domain, Bot Management appears as a tab inside of the Firewall app of the Cloudflare dashboard. You will see a toggle for activating bot management. Once active, Bot Management:
- Allows the cf.bot_management.score variable to be used/manipulated via Firewall Rules and the header Cf-Client-Trust-Score via Workers
- Sets Cloudflare's bot management (__cf__bm__) cookie on your website for score improvement
I enabled Bot Management in Log/Simulate mode. What should I look for?
We recommend that you study the logged data and analyze if any of the information logged resembles your office IPs, your monitoring service, your favorite Python script etc. White list them if necessary.
How does Machine Learning work?
Supervised machine learning takes certain variables (X) like gender and age and predicts another variable (Y) like income. Our X variables are request features. Our Y variable represents the probability of solving a Captcha based on X values. We use data from millions of requests and re-train on a periodic basis. You can learn about this data from your own request logs like Cloudflare Logpull and Logpush as well as the Firewall API.
What is the difference between the threat score and bot score?
The difference is significant:
- Bot score is what Cloudflare uses in Bot Management. It goes from 0 (bad) to 100 (good).
- Threat score is what Cloudflare uses to determine IP Reputation. It goes from 0 (good) to 100 (bad).
What is cf.client.bot?
cf.client.bot is a list of good bots like search engines. Do not block it unless you have a reason to do so.
Going forward, we strongly recommend that you use cf.bot_management.verified_bot instead of cf.client.bot.
Cloudflare has built a white list of good, automated bots, e.g. Google Search Engine, Pingdom, and more. This white list is large based on reverse DNS verification, meaning that the IPs we whitelist really belong to the services you intended.
I'm a good bot and want to be added to the white list (cf.bot_management.verified_bot). What should I do?
To be added to our white list, please submit this online application.