ch35-ai-gateway

取得並執行

這個範例可以獨立 clone 執行，不依賴其他章節。

npx degit mrsuner/cloudflare-serverless-platform-manuals/examples/ch35-ai-gateway
cd ch35-ai-gateway
npm install

可用指令

npm run dev	# wrangler dev
npm run types	# wrangler types --env-interface CloudflareBindings

說明

ch35 — AI Gateway

Probe project for chapter 35.

⚠️ AI Gateway is reached through the AI binding, so it is always remote and bills your account even in wrangler dev (ch34). These routes need a CLOUDFLARE_API_TOKEN.

Run

npm install
npx wrangler types --env-interface CloudflareBindings
export CLOUDFLARE_API_TOKEN=...
npx wrangler dev --port 9044

Routes

Route	What it shows
`/shape`	There is no gateway binding — it hangs off `env.AI`.
`/url`	`getUrl()` per provider.
`/run?q=&tenant=&fresh=1`	Every gateway feature driven from the binding, no headers.
`/cache-proof?k=`	Same `cacheKey` twice; compare latency and the log’s `cached`.
`/feedback?log=&score=&thumb=`	`patchLog()` + `getLog()`.
`/gateway-run`	`gateway.run()` — gone from the docs, still in the types.
`/raw`	The auth-header trap, explained.

No separate binding

const gw = env.AI.gateway("default");

declare abstract class AiGateway {
  patchLog(logId: string, data: AiGatewayPatchLog): Promise<void>;
  getLog(logId: string): Promise<AiGatewayLog>;
  run(data, options?): Promise<Response>;
  getUrl(provider?: AIGatewayProviders | string): Promise<string>;
}

Since 2026-03 the first authenticated request auto-creates a gateway named default, so { gateway: { id: "default" } } works with no setup.

run() is gone from the docs but still present in the shipped types. It compiles; it is not the recommended path. /gateway-run exists so you can check what it currently does.

You never need `cf-aig-*` headers from a Worker

await env.AI.run(model, inputs, {
  gateway: {
    id: "default",
    cacheKey: promptHash,
    cacheTtl: 3600,
    skipCache: false,
    metadata: { tenant: tenantId, feature: "summarise" },
    eventId: crypto.randomUUID(),
    requestTimeoutMs: 20_000,
    retries: { maxAttempts: 3, retryDelayMs: 500, backoff: "exponential" },
  },
});

metadata is the highest-value field — it lands in logs and analytics, so per-tenant, per-feature cost attribution comes for free.

Headers are only needed when you call the gateway URL directly (e.g. with the OpenAI SDK). The shipped AIGatewayHeaders type is the complete list, and includes one the docs barely mention:

'cf-aig-custom-cost': { per_token_in?: number; per_token_out?: number } | { total_cost?: number }

— which overrides the gateway’s own cost estimate. Useful when you have negotiated pricing or a self-hosted model behind the gateway.

Deprecated: cf-cache-ttl, cf-skip-cache (the pre--aig- names).

The auth trap

Endpoint	Cloudflare token goes in	`Authorization` belongs to
`gateway.ai.cloudflare.com/...`	`cf-aig-authorization`	the provider
`api.cloudflare.com/.../ai/...`	`Authorization: Bearer`	Cloudflare

On gateway.ai.cloudflare.com, Authorization is forwarded downstream, so your Cloudflare token must use cf-aig-authorization. Getting this backwards produces a 401 that could be from either side.

Using env.AI.run() sidesteps it entirely.

There is also a two-step deprecation chain in the docs: the Universal Endpoint page is titled “(Deprecated)” and points at the OpenAI-compatible endpoint, which is itself marked deprecated in favour of the REST API. Provider-specific endpoints are not deprecated. From a Worker, none of this matters.

Logs are a free eval dataset

type AiGatewayLog = {
  cached: boolean; cost?: number; custom_cost?: boolean;
  tokens_in?: number; tokens_out?: number; step?: number;
  duration: number; metadata?: Record<...>; /* … */
};

const answer = await env.AI.run(model, inputs, { gateway: { id: "default" } });
const logId = env.AI.aiGatewayLogId;             // this call's log id
// … later, from the user's thumbs-up/down:
await env.AI.gateway("default").patchLog(logId, { feedback: -1, score: 20 });

feedback is typed -1 | 1 | null. The gateway already stores the request and response, so attaching human judgement turns your traffic into an evaluation set with no extra schema.

patchLog() does not verify ownership. A logId round-tripped through the browser is user input — check it belongs to the caller’s tenant first.

20 providers, not 23

AIGatewayProviders in the shipped types lists exactly twenty:

workers-ai anthropic aws-bedrock azure-openai google-vertex-ai huggingface
openai perplexity-ai replicate groq cohere google-ai-studio mistral grok
openrouter deepseek cerebras cartesia elevenlabs adobe-firefly

getUrl(provider?: AIGatewayProviders | string) keeps the | string escape hatch for custom providers (beta).

Know which features are beta

GA: analytics, logging, caching, rate limiting, retries/timeouts, custom cost, OTel export, Logpush.

Beta: spend limit, dynamic routing, DLP, guardrails, BYOK, custom provider, WebSockets.

Two that matter:

Spend limit is beta. If you adopted AI Gateway to cap runaway bills, your first line of defence is still your own code — caching, cheap models, usage logging.
Guardrails costs ~500 ms and disables streaming. For a chat UI that is disqualifying. Call @cf/meta/llama-guard-3-8b yourself on the input only, and keep the output streaming.

Name collision

AI Gateway has no MCP features. MCP Server Portals belong to Cloudflare One / Zero Trust. The names both suggest “a gateway in front of AI services” and send people down the wrong docs tree. See ch37.

`wrangler.jsonc`

{
  "$schema": "./node_modules/wrangler/config-schema.json",
  "name": "ch35-ai-gateway",
  "main": "src/index.ts",
  "compatibility_date": "2026-07-24",
  "observability": { "enabled": true },
  // There is NO separate AI Gateway binding. You reach it through `ai`.
  "ai": { "binding": "AI", "remote": true },
  "d1_databases": [{ "binding": "DB", "database_name": "ch35", "database_id": "ch35-local" }]
}

`package.json`

{
  "name": "ch35-ai-gateway",
  "private": true,
  "type": "module",
  "scripts": { "dev": "wrangler dev", "types": "wrangler types --env-interface CloudflareBindings" },
  "devDependencies": { "wrangler": "^4.118.0", "typescript": "^5.9.2" }
}

`src/index.ts`

const t = async (fn: () => unknown): Promise<Record<string, unknown>> => {
  const started = Date.now();
  try {
    const v = await fn();
    return { ok: v === undefined ? "(undefined)" : v, ms: Date.now() - started };
  } catch (e) {
    return {
      threwName: (e as Error).name,
      threw: String((e as Error).message ?? e).slice(0, 300),
      ms: Date.now() - started,
    };
  }
};

const GATEWAY = "default";

export default {
  async fetch(request: Request, env: CloudflareBindings, ctx: ExecutionContext): Promise<Response> {
    const url = new URL(request.url);
    const q = url.searchParams;

    switch (url.pathname) {
      // There is no `env.GATEWAY`. It hangs off the AI binding.
      case "/shape": {
        const gw = env.AI.gateway(GATEWAY) as unknown as object;
        return Response.json({
          aiProtoKeys: Object.getOwnPropertyNames(Object.getPrototypeOf(env.AI)),
          gatewayCtor: Object.getPrototypeOf(gw)?.constructor?.name ?? null,
          gatewayProtoKeys: Object.getOwnPropertyNames(Object.getPrototypeOf(gw)),
          // Populated after a gateway-routed call; used to patch the log later.
          aiGatewayLogId: (env.AI as { aiGatewayLogId?: unknown }).aiGatewayLogId ?? null,
        });
      }

      // The base URL for a provider, so you can call it with plain fetch().
      case "/url": {
        const gw = env.AI.gateway(GATEWAY);
        return Response.json({
          workersAi: await t(() => gw.getUrl("workers-ai")),
          openai: await t(() => gw.getUrl("openai")),
          noProvider: await t(() => gw.getUrl()),
        });
      }

      /**
       * Everything AI Gateway does can be driven from the BINDING -- you do not
       * need to hand-set cf-aig-* headers when calling through env.AI.run().
       */
      case "/run": {
        const res = await t(() =>
          env.AI.run(
            "@cf/zai-org/glm-4.7-flash",
            { messages: [{ role: "user", content: q.get("q") ?? "one word: hello" }] } as never,
            {
              gateway: {
                id: GATEWAY,
                // Deterministic cache key -> identical prompts hit cache.
                cacheKey: q.get("q") ?? "hello",
                cacheTtl: 3600,
                skipCache: q.get("fresh") === "1",
                // Attribution. Shows up in the log and in analytics.
                metadata: { tenant: q.get("tenant") ?? "acme", feature: "demo" },
                eventId: crypto.randomUUID(),
                requestTimeoutMs: 20_000,
                retries: { maxAttempts: 3, retryDelayMs: 500, backoff: "exponential" },
              },
            },
          ),
        );
        // The log id for THIS call, so we can attach feedback later.
        return Response.json({ logId: env.AI.aiGatewayLogId, result: res });
      }

      // Cache proof: same cacheKey twice, compare latency and the log's `cached`.
      case "/cache-proof": {
        const key = `probe-${q.get("k") ?? "1"}`;
        const call = () =>
          env.AI.run(
            "@cf/zai-org/glm-4.7-flash",
            { messages: [{ role: "user", content: "say exactly: ok" }] } as never,
            { gateway: { id: GATEWAY, cacheKey: key, cacheTtl: 600 } },
          );
        const first = await t(call);
        const firstLog = env.AI.aiGatewayLogId;
        const second = await t(call);
        const secondLog = env.AI.aiGatewayLogId;
        return Response.json({ first, firstLog, second, secondLog });
      }

      // Human feedback loop: score/feedback attach to an existing log entry.
      case "/feedback": {
        const gw = env.AI.gateway(GATEWAY);
        const logId = q.get("log") ?? "";
        return Response.json({
          patched: await t(() =>
            gw.patchLog(logId, {
              score: Number(q.get("score") ?? 80),
              feedback: q.get("thumb") === "down" ? -1 : 1,
              metadata: { source: "ch35" },
            }),
          ),
          log: await t(() => gw.getLog(logId)),
        });
      }

      // `run()` on the gateway object still exists in the TYPES even though it
      // has disappeared from the docs. This route is here to find out whether
      // it still works at runtime.
      case "/gateway-run": {
        const gw = env.AI.gateway(GATEWAY) as unknown as {
          run(data: unknown, options?: unknown): Promise<Response>;
        };
        return Response.json(
          await t(async () => {
            const res = await gw.run({
              provider: "workers-ai",
              endpoint: "@cf/zai-org/glm-4.7-flash",
              headers: { "content-type": "application/json" },
              query: { messages: [{ role: "user", content: "hi" }] },
            });
            return { status: res.status, body: (await res.text()).slice(0, 200) };
          }),
        );
      }

      // Raw fetch through the gateway, showing the auth-header trap.
      case "/raw": {
        const base = await env.AI.gateway(GATEWAY).getUrl("workers-ai");
        return Response.json({
          note:
            "On gateway.ai.cloudflare.com the CLOUDFLARE token goes in cf-aig-authorization; " +
            "Authorization is reserved for the PROVIDER. On api.cloudflare.com it is a plain " +
            "Authorization: Bearer. Mixing these up is the most common 401.",
          base,
        });
      }

      default:
        return new Response(
          "/shape /url /run?q=&tenant=&fresh=1 /cache-proof?k= /feedback?log=&score=&thumb=\n" +
            "/gateway-run /raw\n",
          { status: 404 },
        );
    }
  },
} satisfies ExportedHandler<CloudflareBindings>;

`.gitignore`

node_modules/
.wrangler/
worker-configuration.d.ts

`tsconfig.json`

{
  "compilerOptions": {
    "target": "esnext", "lib": ["esnext"], "module": "esnext",
    "moduleResolution": "bundler", "types": ["./worker-configuration.d.ts"],
    "strict": true, "skipLibCheck": true, "noEmit": true, "isolatedModules": true
  },
  "include": ["src/**/*.ts", "worker-configuration.d.ts"]
}

ch35-ai-gateway

取得並執行

可用指令

說明

ch35 — AI Gateway

Run

Routes

No separate binding

You never need cf-aig-* headers from a Worker

The auth trap

Logs are a free eval dataset

20 providers, not 23

Know which features are beta

Name collision

原始碼

wrangler.jsonc

package.json

src/index.ts

.gitignore

tsconfig.json

You never need `cf-aig-*` headers from a Worker

`wrangler.jsonc`

`package.json`

`src/index.ts`

`.gitignore`

`tsconfig.json`