ch34-workers-ai

對應 34. Workers AI

在 GitHub 上檢視·5 個檔案·9.6 KB

取得並執行

這個範例可以獨立 clone 執行，不依賴其他章節。

npx degit mrsuner/cloudflare-serverless-platform-manuals/examples/ch34-workers-ai
cd ch34-workers-ai
npm install

可用指令

npm run dev	# wrangler dev
npm run types	# wrangler types --env-interface CloudflareBindings

說明

ch34 — Workers AI

Probe project for chapter 34.

⚠️ The AI binding always runs remotely and bills your account, even in wrangler dev. These routes cannot run without a CLOUDFLARE_API_TOKEN.

Verified with wrangler 4.118.0.

Run

npm install
npx wrangler types --env-interface CloudflareBindings
export CLOUDFLARE_API_TOKEN=...
npx wrangler dev --port 9043

Without the token:

env.AI                      AI                remote
⎔ Establishing remote connection...
✘ ERROR Failed to start the remote proxy session ... it's necessary to set a
  CLOUDFLARE_API_TOKEN environment variable

Note wrangler marks the binding remote even though wrangler.jsonc here does not set "remote": true. Set it explicitly anyway, so “this costs money” is visible in the config.

Routes

Route	What it shows
/audit	The important one. Diffs your model IDs against env.AI.models().
/shape	The binding’s surface.
/chat?q=&stream=1	stream: true in the right argument.
/gpt-oss	instructions + input instead of messages.
/affinity?s=	x-session-affinity for prefix-cache hits.
/batch-submit, /batch-poll?id=	The pull-based Batch API.
/third-party	openai/gpt-4.1-mini with and without a gateway.
/dead-model	A deprecated ID and a nonsense ID — both typecheck.
/resilient	Retry policy that distinguishes 3036 from 3040.
/to-markdown	env.AI.toMarkdown().

The types ship dead model IDs

AiModels in the generated worker-configuration.d.ts is a static snapshot. Measured against the 18 IDs Cloudflare announced for removal on 2026-05-30:

Announced deprecated	Still in the generated types?
@cf/meta/llama-3-8b-instruct	yes
@cf/meta/llama-2-7b-chat-fp16	yes
@cf/meta/llama-2-7b-chat-int8	yes
@cf/mistral/mistral-7b-instruct-v0.1	yes
@cf/google/gemma-3-12b-it	yes
@cf/microsoft/phi-2	yes

And the model pinned first in Cloudflare’s own catalogue — @cf/moonshotai/kimi-k2.7-code — is absent from the types (only k2.5 and k2.6 are there, and k2.5 is itself deprecated and auto-aliased to the more expensive k2.6).

So the types are wrong in both directions, and TypeScript will autocomplete a dead ID with no complaint. /dead-model demonstrates that both a deprecated ID and a nonsense ID compile.

Cloudflare’s own docs still print dead IDs too: the Wrangler quickstart, the REST quickstart, 4 of 9 entries in the JSON-mode support list, and the prompt-caching example.

env.AI.models() is the only live source of truth. Run /audit in CI:

const live = await env.AI.models({ per_page: 500 });
const liveNames = new Set(live.map((m) => m.name));
Object.entries(MODELS).map(([k, id]) => [k, { id, live: liveNames.has(id) }]);

Keep every model ID in one MODELS constant, and keep a fallback per role.

Argument positions

Six run() overloads. Which argument an option goes in is not intuitive:

// stream lives in INPUTS (2nd arg)
run(model, inputs & { stream: true }, options?): Promise<ReadableStream>

// queueRequest / websocket / returnRawResponse live in OPTIONS (3rd arg)
run(model, { requests }, options & { queueRequest: true })
run(model, inputs, options & { websocket: true })
run(model, inputs, options & { returnRawResponse: true })

env.AI.run(model, { messages }, { stream: true }) compiles fine and silently does not stream.

3036 vs 3040 are both HTTP 429

Code	Meaning	Retry?
3036	daily free Neuron allocation exhausted	no — it won’t recover today
3040	out of capacity	yes, with backoff

Cloudflare’s errors page is a bare table with no handling guidance at all, so runWithRetry() in src/index.ts is this chapter’s policy, not theirs. Branch on e.code, never on the HTTP status.

Deprecated on the binding itself

/** @deprecated Use the standalone `ai_search_namespaces` or `ai_search` bindings */
aiSearch(): AiSearchNamespace;
/** @deprecated AutoRAG has been replaced by AI Search. */
autorag(autoragId: string): AutoRAG;

If a 2025-era tutorial has env.AI.autorag(...), it’s stale. See ch36.

Three billing surfaces, three prefixes

Prefix	What	Billed as
@cf/...	Workers AI	Neurons
@cf/deepgram/..., @cf/leonardo/...	partner models on Workers AI	Neurons
openai/..., google/...	real third parties	AI Gateway Unified Billing (credits, +5% fee)

Third-party models require { gateway: { id: "default" } }. @cf/ models routed through a gateway still bill as Neurons, not credits. Third-party IDs hit the fallback overload, so inputs and the return are Record<string, unknown> — no type checking at all.

Cost hygiene for local development

Because dev bills the real account:

• develop against the cheapest model (glm-4.7-flash at $0.06/M input vs kimi-k2.7-code at $0.95/M)
• tag requests: { tags: ["env:dev"] } — max 5 tags, 50 chars each, charset letters, numbers, : - . / @
• cache responses in KV; never re-run an identical prompt
• set x-session-affinity on anything with a long system prompt — cached input is ~5× cheaper on the models that support it

原始碼

• wrangler.jsonc
• package.json
• src/index.ts
• .gitignore
• tsconfig.json

wrangler.jsonc

{
  "$schema": "./node_modules/wrangler/config-schema.json",
  "name": "ch34-workers-ai",
  "main": "src/index.ts",
  "compatibility_date": "2026-07-24",
  "observability": { "enabled": true },
  "ai": { "binding": "AI" },
  "kv_namespaces": [{ "binding": "CACHE", "id": "ch34-local" }]
}

package.json

{
  "name": "ch34-workers-ai",
  "private": true,
  "type": "module",
  "scripts": { "dev": "wrangler dev", "types": "wrangler types --env-interface CloudflareBindings" },
  "devDependencies": { "wrangler": "^4.118.0", "typescript": "^5.9.2" }
}

src/index.ts

const t = async (fn: () => unknown): Promise<Record<string, unknown>> => {
  const started = Date.now();
  try {
    const v = await fn();
    return { ok: v === undefined ? "(undefined)" : v, ms: Date.now() - started };
  } catch (e) {
    const err = e as Error & { code?: number };
    return {
      threwName: err.name,
      code: err.code,
      threw: String(err.message ?? e).slice(0, 300),
      ms: Date.now() - started,
    };
  }
};

/**
 * The IDs Cloudflare announced for removal on 2026-05-30. Several of these
 * still appear in `AiModels` in the GENERATED TYPES, and several still appear
 * in Cloudflare's own quickstarts. TypeScript will autocomplete them happily.
 */
const ANNOUNCED_DEPRECATED = [
  "@cf/moonshotai/kimi-k2.5",
  "@hf/meta-llama/meta-llama-3-8b-instruct",
  "@cf/meta/llama-3-8b-instruct",
  "@cf/meta/llama-3-8b-instruct-awq",
  "@cf/meta/llama-3.1-8b-instruct",
  "@cf/meta/llama-3.1-8b-instruct-awq",
  "@cf/meta/llama-3.1-70b-instruct",
  "@cf/meta/llama-2-7b-chat-int8",
  "@cf/meta/llama-2-7b-chat-fp16",
  "@cf/mistral/mistral-7b-instruct-v0.1",
  "@hf/mistral/mistral-7b-instruct-v0.2",
  "@hf/google/gemma-7b-it",
  "@cf/google/gemma-3-12b-it",
  "@hf/nousresearch/hermes-2-pro-mistral-7b",
  "@cf/microsoft/phi-2",
  "@cf/defog/sqlcoder-7b-2",
  "@cf/unum/uform-gen2-qwen-500m",
  "@cf/facebook/bart-large-cnn",
] as const;

/** The model IDs this app actually uses. Keep this list short and audited. */
const MODELS = {
  chat: "@cf/meta/llama-4-scout-17b-16e-instruct",
  cheap: "@cf/zai-org/glm-4.7-flash",
  embed: "@cf/google/embeddinggemma-300m",
  guard: "@cf/meta/llama-guard-3-8b",
} as const;

export default {
  async fetch(request: Request, env: CloudflareBindings, ctx: ExecutionContext): Promise<Response> {
    const url = new URL(request.url);
    const q = url.searchParams;

    switch (url.pathname) {
      case "/shape": {
        const b = env.AI as unknown as object;
        return Response.json({
          ctorName: Object.getPrototypeOf(b)?.constructor?.name ?? null,
          protoKeys: Object.getOwnPropertyNames(Object.getPrototypeOf(b)),
          ownKeys: Object.getOwnPropertyNames(b),
          aiGatewayLogId: (env.AI as { aiGatewayLogId?: unknown }).aiGatewayLogId ?? null,
        });
      }

      /**
       * THE MOST USEFUL ROUTE HERE.
       *
       * `AiModels` in worker-configuration.d.ts is a static snapshot. It ships
       * dead IDs and can lag new ones. `env.AI.models()` is the live catalogue.
       * Run this in CI to fail the build when a model you depend on disappears.
       */
      case "/audit": {
        return Response.json(
          await t(async () => {
            const live = await env.AI.models({ per_page: 500 });
            const liveNames = new Set(live.map((m) => m.name));

            return {
              liveModelCount: live.length,
              // Are the models this app uses still real?
              inUse: Object.fromEntries(
                Object.entries(MODELS).map(([k, id]) => [k, { id, live: liveNames.has(id) }]),
              ),
              // Which announced-deprecated IDs are still being served?
              deprecatedStillLive: ANNOUNCED_DEPRECATED.filter((id) => liveNames.has(id)),
              // Which IDs does the type system know that the API does not?
              // (Fill TYPED_IDS from your own AiModels keys if you want this.)
            };
          }),
        );
      }

      // `stream: true` goes in INPUTS (2nd arg), not options (3rd).
      case "/chat": {
        const stream = q.get("stream") === "1";
        const inputs = {
          messages: [
            { role: "system", content: "Answer in one sentence." },
            { role: "user", content: q.get("q") ?? "What is a Cloudflare Worker?" },
          ],
          max_tokens: 200,
          ...(stream ? { stream: true as const } : {}),
        };

        if (stream) {
          const body = await env.AI.run(MODELS.chat, inputs as never);
          return new Response(body as ReadableStream, {
            headers: { "content-type": "text/event-stream" },
          });
        }
        return Response.json(await t(() => env.AI.run(MODELS.chat, inputs as never)));
      }

      // gpt-oss takes `instructions` + `input`, not `messages`.
      case "/gpt-oss": {
        return Response.json(
          await t(() =>
            env.AI.run("@cf/openai/gpt-oss-120b", {
              instructions: "You are a concise assistant.",
              input: q.get("q") ?? "Explain V8 isolates in one sentence.",
            } as never),
          ),
        );
      }

      // Prompt caching depends on landing on the same model instance.
      case "/affinity": {
        const session = q.get("s") ?? "ses_ch34";
        const longPrefix = "You are LinkForge's assistant. ".repeat(50);
        return Response.json(
          await t(() =>
            env.AI.run(
              MODELS.cheap,
              {
                messages: [
                  { role: "system", content: longPrefix },
                  { role: "user", content: q.get("q") ?? "hi" },
                ],
              } as never,
              // Documented mechanism for prefix-cache hits.
              { extraHeaders: { "x-session-affinity": session }, tags: ["ch34", "affinity"] },
            ),
          ),
        );
      }

      // Batch: submit with queueRequest, then poll with request_id.
      case "/batch-submit": {
        return Response.json(
          await t(() =>
            env.AI.run(
              MODELS.embed,
              { requests: [{ text: "alpha" }, { text: "beta" }, { text: "gamma" }] } as never,
              { queueRequest: true },
            ),
          ),
        );
      }
      case "/batch-poll": {
        return Response.json(
          await t(() =>
            env.AI.run(MODELS.embed, { request_id: q.get("id") ?? "" } as never),
          ),
        );
      }

      // Third-party models MUST carry a gateway.
      case "/third-party": {
        return Response.json({
          withGateway: await t(() =>
            env.AI.run(
              "openai/gpt-4.1-mini",
              { messages: [{ role: "user", content: "one word: hello" }] },
              { gateway: { id: q.get("gw") ?? "default" } },
            ),
          ),
          withoutGateway: await t(() =>
            env.AI.run("openai/gpt-4.1-mini", {
              messages: [{ role: "user", content: "one word: hello" }],
            }),
          ),
        });
      }

      // A model ID that does not exist -- what does the platform say?
      case "/dead-model": {
        return Response.json({
          note: "Both of these typecheck. AiModels still contains the first one.",
          deprecated: await t(() =>
            env.AI.run("@cf/meta/llama-3-8b-instruct", {
              messages: [{ role: "user", content: "hi" }],
            } as never),
          ),
          nonsense: await t(() =>
            env.AI.run("@cf/not/a-real-model" as never, { prompt: "hi" } as never),
          ),
        });
      }

      // Retry wrapper for 3040 (out of capacity). 3036 must NOT be retried.
      case "/resilient": {
        return Response.json(
          await t(() =>
            runWithRetry(env, MODELS.cheap, {
              messages: [{ role: "user", content: q.get("q") ?? "hi" }],
            }),
          ),
        );
      }

      case "/to-markdown": {
        return Response.json(
          await t(() =>
            env.AI.toMarkdown([
              {
                name: "note.txt",
                blob: new Blob(["# Hello\n\nplain text input"], { type: "text/plain" }),
              },
            ]),
          ),
        );
      }

      default:
        return new Response(
          "/shape /audit /chat?q=&stream=1 /gpt-oss /affinity /batch-submit /batch-poll?id=\n" +
            "/third-party /dead-model /resilient /to-markdown\n",
          { status: 404 },
        );
    }
  },
} satisfies ExportedHandler<CloudflareBindings>;

/**
 * 3036 = daily free Neuron allocation exhausted. Retrying is pointless.
 * 3040 = out of capacity. This one IS worth retrying, with backoff.
 * Cloudflare's errors page is a bare table with no handling guidance at all,
 * so this policy is ours.
 */
async function runWithRetry(
  env: CloudflareBindings,
  model: Parameters<typeof env.AI.run>[0],
  inputs: unknown,
  attempts = 4,
): Promise<unknown> {
  let lastError: unknown;
  for (let i = 0; i < attempts; i++) {
    try {
      return await env.AI.run(model as never, inputs as never);
    } catch (e) {
      const code = (e as { code?: number }).code;
      if (code === 3036) throw e;            // out of free allocation -- permanent today
      if (code !== 3040 && code !== undefined) throw e;
      lastError = e;
      await new Promise((r) => setTimeout(r, 250 * 2 ** i));   // 250/500/1000/2000ms
    }
  }
  throw lastError;
}

.gitignore

node_modules/
.wrangler/
worker-configuration.d.ts

tsconfig.json

{
  "compilerOptions": {
    "target": "esnext", "lib": ["esnext"], "module": "esnext",
    "moduleResolution": "bundler", "types": ["./worker-configuration.d.ts"],
    "strict": true, "skipLibCheck": true, "noEmit": true, "isolatedModules": true
  },
  "include": ["src/**/*.ts", "worker-configuration.d.ts"]
}