New at ContentCon: meet Contentstack Canoe. See what AI says about your brand.
EnterpriseIntegrationCloudflare Workers

Connect Cloudflare

Contentstack Canoe installs a small Worker on your site's Cloudflare zone with an API token you create in Cloudflare. It reports which AI agents read your pages. People get your site exactly as before.

Enterprise feature

Agent traffic is part of the Enterprise plan. To turn it on for your workspace, talk to us.

What the Worker does

  • People pass straight through. Their requests go to your origin untouched. When someone arrives from a link in an AI answer, the page and the assistant are counted, with no IP address, user agent or cookies.
  • AI agent visits are reported. Agents get your normal page. After the response is on its way, the Worker sends Contentstack Canoe a short record: the time, host and path (without the query string), the agent's user agent and signature headers, how your site answered, and where the request came from: Cloudflare data center, network and country, plus city and region for agent requests only. When Cloudflare marks a request as a verified bot, or your zone has Bot Management, those signals are sent too.
  • Private data stays private. Cookies, request bodies and form data are never read or sent. The IP address is used to check the agent, and only its network prefix is stored. The Worker sets no cookies and keeps Workers Logs off.
  • It fails open. If the Worker hits an error, or Contentstack Canoe is slow or down, the request goes to your origin as usual.
  • Pages stay the same. AI agents get a different version only if you turn on agent-ready pages. They're off by default.

Before you start

  • Your site on a Cloudflare zone, with the DNS record for each host proxied (orange cloud). Runs on a platform's Cloudflare zone instead? See Sites on a platform's zone. Not on Cloudflare? Use Custom edge.
  • Workers on your Cloudflare plan. The Worker runs on every request to the hosts you pick, people included, so each request counts toward your Workers usage. A paid Workers plan has no daily request limit.
  • Someone who can create API tokens in that Cloudflare account. The Token step has Copy steps for your admin.
  • Owner or admin in Contentstack Canoe to connect, change or disconnect. Everyone in the workspace can see the install.

Create the API token

Recommended for enterprise accounts
  1. Install with an account-owned token and a short expiry, such as one day.
  2. Right after install, select Switch to a maintenance token on the Cloudflare card. That token reaches only the installed contentstack-canoe- Worker, plus the zone permissions.
  3. Delete the install token in Cloudflare.

The install token is account-scoped: creating a Worker needs Developer Platform > Workers > Admin, which covers every Worker in the account. Cloudflare only lets a token be limited to one Worker once that Worker exists, which is why the maintenance token comes second.

Three permissions are required. The optional ones each turn on one extra view, and leaving one out only hides that view. The names below are how Cloudflare's account token form shows them.

Required permissions

Required token permissions and why each one is needed
PermissionWhyNeeded
Developer Platform > Workers > AdminCreate, update and remove the Contentstack Canoe Worker. Editor can't create or remove a Worker. It covers every Worker in the account, but Contentstack Canoe only changes the one it tagged as its own.Required
Developer Platform > Workers Routes > EditAdd and remove the routes that run the Worker on your hosts.Required
DNS & Zones > Zone > ReadFind the zone and account that serve your site.Required

Optional permissions

Optional token permissions and what each one turns on
PermissionWhat it turns onNeeded
Analytics & Logs > Account Analytics > ReadBefore you install: the AI crawlers Cloudflare saw on your site in the last 7 days, and how much traffic the Worker would run on.Optional
Analytics & Logs > Zone Analytics > ReadHow many AI agent requests Cloudflare blocked or challenged before they reached your site.Optional
App Security > Bot Management > ReadYour Cloudflare AI crawler setting (allow, block or charge) next to each agent.Optional
DNS & Zones > Zone Settings > ReadWhether Cloudflare already serves markdown to agents for your zone, so it isn't done twice, and whether pay per crawl is on.Optional
Cache & Performance > Zone SSL & Certificates > ReadFinds your site when it runs on a platform's Cloudflare zone as a custom hostname. Read only: certificates are never changed.Optional
  1. On the Token step, select Create token in Cloudflare. Cloudflare's form opens with the name Contentstack Canoe (your domain) and every permission above already picked. Remove any optional one you don't want.

  2. On the zone permissions, select Edit policy and keep only your site's zone. On a personal token, that's Zone Resources.

  3. Select Review token, then Create Token, and copy it. Cloudflare shows it only once.

  • Use an account token, the default. It keeps working when the person who made it leaves.
  • Set a short expiry if you'll switch to a maintenance token. Contentstack Canoe warns you two weeks before any token ends.

If your form doesn't offer Workers > Admin, use the legacy Developer Platform > Workers Scripts > Edit. A personal token's form names a few permissions slightly differently, such as Analytics for Zone Analytics and SSL and Certificates, but the link picks them for you either way.

Contentstack Canoe encrypts the token and uses it only to install, check, update and remove its Worker.

Connect

Open your brand's settings, select Integrations, then Connect on the Cloudflare card. Four steps follow: Token, Hostnames, Review and Verify.

  1. Token. Create the token from the link on this step, paste it and select Check token. Token tips and why each permission explains each one and has Copy steps for your admin.

  2. Hostnames. Check the zone and the hostnames to watch, such as brightkeel.example and www.brightkeel.example. Each hostname shows the zone it runs on. Each one gets a Worker route, like www.brightkeel.example/*. To try it on staging first, add only the staging hostname now and the rest later.

  3. Review. The list shows every change Contentstack Canoe will make. Nothing changes until you select Install in observe-only.

  4. Verify. The first test request usually arrives in under a minute, and the page says Cloudflare is connected.

Connect Cloudflare, step 1 of 4: Create token in Cloudflare, What the token needs, and the field to paste the token.
Step 1. The link picks every permission for you.
Step 2 for the fictional brand Brightkeel: the zones brightkeel.example and harborhost-sites.net picked, and three hostnames, two on brightkeel.example and www.brightkeelshop.example through harborhost-sites.net.
Step 2 for a fictional brand. Each hostname shows where it runs.
Step 3, Review what will change: the hostnames, one route per hostname on the zone that serves it, then Install in observe-only.
Step 3. The install starts in observe-only: it reports visits and changes nothing visitors get.
  • See AI crawlers first. With Account Analytics Read, Show AI crawler visits on the Hostnames step lists the AI crawlers Cloudflare logged on your site in the last 7 days, including ones your security rules blocked.
  • Agent-ready pages stay off until you turn them on in Agent Traffic.

Sites on a platform's zone

Some sites run on a platform, such as a hosting service, that serves them from the platform's own Cloudflare zone. Your hostname is then a custom hostname on that zone (Cloudflare calls this Cloudflare for SaaS), not a zone of its own.

  • Found for you. When your hostname points to the platform's zone, Contentstack Canoe picks that zone and your hostname on the Hostnames step. The hostname shows through the platform's zone.
  • The token comes from the platform's account. The route goes on the platform's zone, so whoever runs that Cloudflare account creates the token. Ask the platform's team to connect, or to create the token for you.
  • Add the SSL read permission. Cache & Performance > Zone SSL & Certificates > Read lets Contentstack Canoe find custom hostnames. It's read only.
  • Only your hostname is watched. The route names your hostname, like www.brightkeelshop.example/*, so other sites on the platform's zone are never seen.
  • The platform's Worker keeps running. Platforms usually run their own Worker on the zone. The review then asks for your consent to pass every request on to it unchanged, as described below.

If your hosts already run a Worker

The review says Needs your consent. If you agree, the Contentstack Canoe Worker runs first and passes every request on to your Worker unchanged. In this setup it reports visits only and doesn't serve agent-ready pages. Every host must run that same Worker.

What changes in Cloudflare

Only what the review lists. DNS, caching, security rules and your other Workers are not touched.

Worker
contentstack-canoe- plus 8 characters, tagged as Contentstack Canoe's own.
Routes
One per host, like www.brightkeel.example/*. With your consent, your existing route points to the Contentstack Canoe Worker instead.
Test requests
A few to your home page after install, then one with each health check, about once an hour. They use the user agent ContentstackCanoeEdgeCheck, not an AI agent's, so AI bot rules don't block them.

Check the install

The Cloudflare card in Integrations shows what's installed and a row for each check: API token, Worker, Routes, Test request and Worker version. They run about once an hour, and Check now runs them right away.

The Cloudflare details on the card: install, account, Worker, routes and version, then checks for the API token, Worker, routes, test request and Worker version, all passing.
A healthy install on a fictional site.

If your zone has Cloudflare Bot Management, the card says so. Verification then uses Cloudflare's own verdict, and those visits show as Verified by Cloudflare in Agent Traffic.

To see a visit yourself, request a page with an AI agent's user agent:

curl -A "Mozilla/5.0 (compatible; GPTBot/1.3; +https://openai.com/gptbot)" https://www.brightkeel.example/pricing

It shows in Agent Traffic within a few minutes, marked Spoofed or Unverified because it doesn't come from OpenAI's network. That's expected. A normal browser visit doesn't show at all.

Update Worker

When a new version is out, the Worker version row says Update available. Update Worker uploads it with the same settings and secret. Nothing else changes.

Turn on optional features later

The Optional features section of the Cloudflare card shows which ones are on. For one that's off, How to turn it on gives the exact steps. Nothing needs to be reinstalled.

  1. In Cloudflare, open the token: Manage Account > Account API Tokens for an account token, or My Profile > API Tokens for a personal one. Select Edit.

  2. Add the permission and save. Analytics & Logs > Account Analytics > Read applies to the account; the others apply to your site's zone. The token value stays the same, so there's nothing to paste.

  3. In Contentstack Canoe, select Check again under Optional features on the Cloudflare card.

Optional features on the Cloudflare card: Blocked by Cloudflare on, AI Crawl Control status needing Zone > Bot Management > Read with the steps to add it, Markdown for Agents check on, and Check again.
One feature off, with its steps open.

In Agent Traffic, Turn on more Cloudflare insights takes you to this section.

Use a maintenance token

Recommended. The maintenance token is the least-privilege setup: it can update and remove only the Contentstack Canoe Worker. Under API token on the Cloudflare card, select Switch to a maintenance token. The card lists the steps with your Worker's name.

Create the maintenance token
  1. In Cloudflare, open Manage Account > Account API Tokens and select Create Token.

  2. Name it Contentstack Canoe maintenance (your domain).

  3. Under Permission policies, set the scope to Specified Workers and pick your contentstack-canoe- Worker. Under Individual Workers, choose Admin. Editor can't remove the Worker later.

  4. Add Developer Platform > Workers Routes > Edit for your zone.

  5. Paste the token, confirm you chose Admin and select Check and switch.

Contentstack Canoe checks that the token can manage only this Worker, without changing anything your visitors see. Then delete the install token in Cloudflare. Add any optional permissions you use to this token too. Not available yet when Contentstack Canoe runs in front of your own Worker.

Rotate the secret

The Worker signs its reports with a secret. Rotate secret makes a new one and writes it to your Worker. The old one keeps working for 48 hours, so no visits are lost. If it leaked, tick Stop accepting the old secret right away.

Replace the token

Token expiring, revoked or too broad? Select Replace token and paste one with the same permissions. Contentstack Canoe checks it can reach the Worker before it swaps. The Worker isn't changed. Then delete the old token in Cloudflare.

Remove the Worker

Disconnect on the Cloudflare card lists what it removes before you confirm: the Worker, its routes and the stored token. A route that passed requests on to your own Worker points back to it. Agent traffic already collected stays in your reports. Then delete the token in Cloudflare.

If Cloudflare can't be reached or the token was revoked, the install moves to Needs cleanup at the top of Integrations with the exact steps left. Select Try again with a new token, which is used once and never stored. Or do the steps in Cloudflare and select I removed it. Until then the Worker keeps passing requests through, and Contentstack Canoe ignores what it sends.

Troubleshooting

While you connect

“Cloudflare did not accept this token.”

Fix: copy the whole token again, the value and not its name. If it expired, was disabled or was deleted, create a new one.

“This token is missing a permission”

Fix: open Details under the message to see which one. Add it to the token in Cloudflare, then try again.

“Cloudflare did not let this token look up its account.”

Fix: add Account & Billing > Account Settings > Read to the token in Cloudflare. It applies to the account, not the zone. Then try again.

“No zone found”

Fix: give the token DNS & Zones > Zone > Read on your site's zone. If your site runs on a platform's zone, the token has to come from that platform's Cloudflare account.

“… isn't on the zones you picked”

Fix: pick the zone that serves that hostname, or select Remove it. If it runs on a platform's zone as a custom hostname, pick the platform's zone and add Cache & Performance > Zone SSL & Certificates > Read to the token so it can be found.

The hostnames list: three hostnames ready and blog.brightkeel.io marked Not on these zones, with the message blog.brightkeel.io isn't on the zones you picked, Remove it and Details.
One line, its fix and the reason under Details.
“… isn't active in Cloudflare yet”

Fix: the custom hostname is still pending on the platform's zone. Select Remove for now and add it once Cloudflare shows it as active.

“The zone … is paused in Cloudflare”

Fix: Workers don't run on a paused or unfinished zone. Resume it, or finish setting it up, in Cloudflare.

“This site is already monitored by another brand”

Fix: disconnect Cloudflare from that brand first. If it's in another workspace, ask an owner there or contact support.

“This site still runs …, a Contentstack Canoe Worker left from an earlier connection”

Fix: finish the steps under Needs cleanup, then review the plan again.

“Your Cloudflare setup changed since you reviewed the plan.”

Fix: select Review again.

“This Cloudflare account has reached its Workers limit.”

Fix: delete a Worker you no longer use, or raise the limit with Cloudflare.

“Cloudflare is limiting requests from this token right now.”

Fix: wait a few minutes, then try again.

After install

“Still waiting for the test request”

Fix: in Cloudflare, check that:

  • each host's DNS record is proxied (orange cloud)
  • no more specific route or other Worker takes the request first
  • WAF, Bot Fight Mode or AI traffic rules don't block it
“Contentstack Canoe can no longer reach your Cloudflare account.”

Fix: the token was revoked or expired. The Worker still runs. Replace the token.

“The Contentstack Canoe Worker is missing from your Cloudflare account.”

Fix: disconnect Cloudflare and connect it again. The same fix works when a route was removed, or when your own Worker behind it is gone.

Fewer AI agent visits than the preview showed

Likely cause: WAF rules, Bot Fight Mode and AI Crawl Control stop some agents before any Worker runs. The preview counts those as blocked or challenged.