cloudflare · kathayl · Mar 16, 2026
@@ -136,14 +136,16 @@ If you are attempting to scan your own zone and want Browser Rendering to access
 
 You must be on an Enterprise plan to allowlist Browser Rendering on your own website because WAF custom rules require access to [Bot Management](/bots/get-started/bot-management/) fields.
 
+Browser Rendering uses different [bot detection IDs](/browser-rendering/reference/automatic-request-headers/#bot-detection) depending on the method. Use the ID that matches the method you want to allowlist.
+
 <Steps>
 1. In the Cloudflare dashboard, go to the **Security rules** page of your account and domain.
 
      <DashButton url="/?to=/:account/:zone/security/security-rules" />
 
 2. To create a new empty rule, select **Create rule** > **Custom rules**.
 3. Enter a descriptive name for the rule in **Rule name**, such as `Allow Browser Rendering`.
-4. Under **When incoming requests match**, use the **Field** dropdown to choose _Bot Detection ID_. For **Operator**, select _equals_. For **Value**, enter `128292352`.
+4. Under **When incoming requests match**, use the **Field** dropdown to choose _Bot Detection ID_. For **Operator**, select _equals_. For **Value**, enter the [bot detection ID](/browser-rendering/reference/automatic-request-headers/#bot-detection) for the method you want to allowlist.
 5. Under **Then take action**, in the **Choose action** dropdown, select  **Skip**.
 6. Under **Place at**, select the order of the rule in the **Select order** dropdown to be **First**. Setting the order as **First** allows this rule to be applied before subsequent rules.
 7. To save and deploy your rule, select **Deploy**.

@@ -11,18 +11,16 @@ Cloudflare automatically attaches headers to every request made through Browser
 
 The default User-Agent depends on how you access Browser Rendering:
 
-| Method | Default User-Agent |
-| --- | --- |
-| [REST API](/browser-rendering/rest-api/) | `Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36` |
-| [Crawl endpoint](/browser-rendering/rest-api/crawl-endpoint/) | `CloudflareBrowserRenderingCrawler/1.0` |
-| [Workers Bindings](/browser-rendering/workers-bindings/) | The default User-Agent of the underlying Chrome version |
+| Method | Default User-Agent | Customizable |
+| --- | --- | --- |
+| [REST API](/browser-rendering/rest-api/) | `Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36` | Yes, using the `userAgent` parameter |
+| [Crawl endpoint](/browser-rendering/rest-api/crawl-endpoint/) | `CloudflareBrowserRenderingCrawler/1.0` | No |
+| [Workers Bindings](/browser-rendering/workers-bindings/) | The default User-Agent of the underlying Chrome version | Yes, via Puppeteer/Playwright configuration |
 
 :::note
-The Chrome version in the REST API and Workers Bindings User-Agent strings may change as Browser Rendering updates its underlying browser engine. Do not rely on a specific version number.
+Because the User-Agent is configurable for most methods and the Chrome version may change as Browser Rendering updates its underlying browser engine, destination servers should use the non-configurable headers below to identify Browser Rendering requests rather than relying on the User-Agent string.
 :::
 
-Unlike the headers listed below, the User-Agent can be customized for the [REST API](/browser-rendering/rest-api/) (using the `userAgent` parameter) and [Workers Bindings](/browser-rendering/workers-bindings/) configuration. The [crawl endpoint](/browser-rendering/rest-api/crawl-endpoint/) is the exception. Its User-Agent is always `CloudflareBrowserRenderingCrawler/1.0` and cannot be changed. Because the User-Agent is configurable for most methods, destination servers should use the headers below to identify Browser Rendering requests rather than relying on the User-Agent string.
-
 ## Non-configurable headers
 
 :::note
@@ -43,4 +41,11 @@ The `Signature` headers use an authentication method called [Web Bot Auth](/bots
 
 ### Bot detection
 
-The bot detection ID for Browser Rendering is `128292352`. If you are attempting to scan your own zone and want Browser Rendering to access your website freely without your bot protection configuration interfering, you can create a WAF skip rule to [allowlist Browser Rendering](/browser-rendering/faq/#can-i-allowlist-browser-rendering-on-my-own-website).
+Browser Rendering uses different bot detection IDs depending on the method. The [REST API](/browser-rendering/rest-api/) (excluding the [crawl endpoint](/browser-rendering/rest-api/crawl-endpoint/)) and [Workers Bindings](/browser-rendering/workers-bindings/) share one ID, while the crawl endpoint has its own.
+
+| Method | Bot detection ID |
+| --- | --- |
+| [REST API](/browser-rendering/rest-api/) and [Workers Bindings](/browser-rendering/workers-bindings/) | `119853733` |
+| [Crawl endpoint](/browser-rendering/rest-api/crawl-endpoint/) | `128292352` |
+
+If you are attempting to scan your own zone and want Browser Rendering to access your website freely without your bot protection configuration interfering, you can create a WAF skip rule to [allowlist Browser Rendering](/browser-rendering/faq/#can-i-allowlist-browser-rendering-on-my-own-website).
@@ -11,10 +11,11 @@ This page provides general guidance on configuring `robots.txt` and sitemaps for
 
 Requests can be identified by the [automatic headers](/browser-rendering/reference/automatic-request-headers/) that Cloudflare attaches:
 
+- [User-Agent](/browser-rendering/reference/automatic-request-headers/#user-agent) — Each Browser Rendering method has a different default User-Agent, which you can use to write targeted `robots.txt` rules
 - `cf-brapi-request-id` — Unique identifier for REST API requests
 - `Signature-agent` — Pointer to Cloudflare's bot verification keys
 
-Browser Rendering has a bot detection ID of `128292352`. Use this to create WAF rules that allow or block Browser Rendering traffic. For the default user agent and other identification details, refer to [Automatic request headers](/browser-rendering/reference/automatic-request-headers/).
+To allow or block Browser Rendering traffic using WAF rules instead of `robots.txt`, use the [bot detection IDs](/browser-rendering/reference/automatic-request-headers/#bot-detection) on the automatic request headers page.
 
 ## Best practices for robots.txt
 
@@ -82,9 +83,9 @@ User-agent: *
 Allow: /
 ```
 
-### Block specific paths
+### Block the /crawl endpoint on specific paths
 
-To allow crawling of your site but block specific sections:
+To allow the [`/crawl` endpoint](/browser-rendering/rest-api/crawl-endpoint/) to access your site but block specific sections:
 
 ```txt title="robots.txt"
 User-agent: CloudflareBrowserRenderingCrawler

@@ -430,7 +430,7 @@ The `/crawl` endpoint respects the directives of `robots.txt` files, including `
 :::caution[Bot protection may block crawling]
 Browser Rendering does not bypass CAPTCHAs, Turnstile challenges, or any other bot protection mechanisms. If a target site uses Cloudflare products that control or restrict bot traffic such as [Bot Management](/bots/), [Web Application Firewall (WAF)](/waf/), or [Turnstile](/turnstile/), the same rules will apply to the Browser Rendering crawler.
 
-If you are crawling your own site and want Browser Rendering to access it freely, you can create a WAF skip rule to allowlist Browser Rendering. Refer to [How do I allowlist Browser Rendering?](/browser-rendering/faq/#can-i-allowlist-browser-rendering-on-my-own-website) for instructions.
+If you are crawling your own site and want Browser Rendering to access it freely, you can create a WAF skip rule to allowlist Browser Rendering. Refer to [How do I allowlist Browser Rendering?](/browser-rendering/faq/#can-i-allowlist-browser-rendering-on-my-own-website) for instructions. The `/crawl` endpoint uses [bot detection ID](/browser-rendering/reference/automatic-request-headers/#bot-detection) `128292352`.
 :::
 
 ### User-Agent