🤖 feat: add Cloudflare AI Gateway support

bun.lock

@@ -1,5 +1,6 @@
 {
   "lockfileVersion": 1,
+  "configVersion": 0,
   "workspaces": {
     "": {
       "name": "@coder/cmux",

docs/SUMMARY.md

@@ -26,6 +26,7 @@
 
 - [Prompting Tips](./prompting-tips.md)
 - [System Prompt](./system-prompt.md)
+- [Cloudflare AI Gateway](./cloudflare-ai-gateway.md)
 - [Telemetry](./telemetry.md)
 
 # Development

docs/cloudflare-ai-gateway.md

@@ -0,0 +1,167 @@
+# Cloudflare AI Gateway
+
+Cloudflare AI Gateway provides a unified interface for accessing AI models from various providers with built-in features like caching, rate limiting, analytics, and cost control.
+
+## What is Cloudflare AI Gateway?
+
+Cloudflare AI Gateway sits between your application and AI model providers, offering:
+
+- **Request management**: Rate limiting, request queueing, and load balancing
+- **Caching**: Cache AI responses to reduce costs and improve latency
+- **Analytics**: Track usage, costs, and performance across all providers
+- **Cost control**: Set spending limits and monitor API usage
+- **Observability**: Detailed logs of all AI requests and responses
+
+## Setup
+
+### 1. Create a Cloudflare AI Gateway
+
+1. Sign up for a [Cloudflare account](https://dash.cloudflare.com/sign-up) (free tier available)
+2. Navigate to the AI Gateway section in your Cloudflare dashboard
+3. Click "Create Gateway" and give it a name (e.g., `my-mux-gateway`)
+4. Note your **Account ID** (found in the dashboard URL or account settings)
+5. Note your **Gateway Name** (the name you just created)
+
+### 2. Configure mux
+
+Open or create `~/.mux/providers.jsonc` and add the `cloudflareGateway` configuration to any provider:
+
+```jsonc
+{
+  "anthropic": {
+    "apiKey": "sk-ant-...",
+    "cloudflareGateway": {
+      "accountId": "your-cloudflare-account-id",
+      "gatewayName": "my-mux-gateway",
+    },
+  },
+  "openai": {
+    "apiKey": "sk-...",
+    "cloudflareGateway": {
+      "accountId": "your-cloudflare-account-id",
+      "gatewayName": "my-mux-gateway",
+    },
+  },
+}
+```
+
+### 3. Verify Setup
+
+When you send a message, mux will automatically route requests through Cloudflare AI Gateway. You should see a log message in the console:
+
+```
+Using Cloudflare AI Gateway { provider: 'anthropic', accountId: '...', gatewayName: '...' }
+```
+
+All AI requests will now appear in your Cloudflare AI Gateway dashboard with detailed analytics.
+
+## Configuration Options
+
+### Per-Provider Configuration
+
+You can configure different gateways for different providers:
+
+```jsonc
+{
+  "anthropic": {
+    "apiKey": "sk-ant-...",
+    "cloudflareGateway": {
+      "accountId": "account-id-1",
+      "gatewayName": "production-gateway",
+    },
+  },
+  "openai": {
+    "apiKey": "sk-...",
+    "cloudflareGateway": {
+      "accountId": "account-id-2",
+      "gatewayName": "testing-gateway",
+    },
+  },
+  // Ollama without gateway (direct connection)
+  "ollama": {
+    "baseUrl": "http://localhost:11434/api",
+  },
+}
+```
+
+### Mixing Gateway and Direct Connections
+
+You can use Cloudflare AI Gateway for some providers and direct connections for others. Simply omit the `cloudflareGateway` configuration for providers you want to connect directly.
+
+## How It Works
+
+When `cloudflareGateway` is configured:
+
+1. mux constructs the Cloudflare AI Gateway URL:
+
+   ```
+   https://gateway.ai.cloudflare.com/v1/{accountId}/{gatewayName}/{provider}
+   ```
+
+2. This URL is passed to the Vercel AI SDK as the `baseURL` parameter
+
+3. All API requests are routed through Cloudflare's infrastructure
+
+4. Your provider API key is still required and sent with each request
+
+5. Cloudflare logs, caches, and manages the requests according to your gateway settings
+
+## Benefits
+
+### Cost Optimization
+
+- **Caching**: Identical requests can be served from cache, avoiding provider API calls
+- **Rate limiting**: Prevent unexpected bill spikes from runaway requests
+- **Usage tracking**: Monitor spending across all providers in one dashboard
+
+### Performance
+
+- **Global edge network**: Requests are routed through Cloudflare's global network
+- **Intelligent caching**: Reduce latency for repeated queries
+- **Load balancing**: Distribute requests efficiently
+
+### Observability
+
+- **Request logs**: See every AI request and response
+- **Analytics**: Track usage patterns, error rates, and performance
+- **Cost breakdown**: Understand spending by model, project, or time period
+
+## Troubleshooting
+
+### Gateway Not Working
+
+1. **Verify account ID and gateway name**: Check your Cloudflare dashboard
+2. **Check API key**: Ensure your provider API key is still valid
+3. **Review Cloudflare logs**: Check the AI Gateway dashboard for error messages
+4. **Test direct connection**: Temporarily remove `cloudflareGateway` to verify basic connectivity
+
+### Performance Issues
+
+1. **Check cache settings**: Cloudflare might be caching responses you don't want cached
+2. **Review rate limits**: Your gateway might be throttling requests
+3. **Check regional routing**: Ensure Cloudflare edge locations are optimal for your region
+
+### Missing Analytics
+
+1. **Verify gateway is active**: Check Cloudflare dashboard shows "Active" status
+2. **Wait for propagation**: Analytics may take a few minutes to appear
+3. **Check configuration**: Ensure `accountId` and `gatewayName` match exactly
+
+## Supported Providers
+
+Cloudflare AI Gateway supports all major AI providers that mux integrates with:
+
+- Anthropic (Claude)
+- OpenAI (GPT, o1, o3)
+- Google (Gemini)
+- Any provider using OpenAI-compatible APIs
+
+## Advanced Configuration
+
+For advanced Cloudflare AI Gateway features (custom caching, rate limiting, etc.), configure them in your Cloudflare dashboard. The mux integration automatically uses whatever settings you've configured for your gateway.
+
+## Learn More
+
+- [Cloudflare AI Gateway Documentation](https://developers.cloudflare.com/ai-gateway/)
+- [Vercel AI SDK Integration Guide](https://developers.cloudflare.com/ai-gateway/integrations/vercel-ai-sdk/)
+- [Cloudflare AI Gateway Dashboard](https://dash.cloudflare.com/)

src/node/config.ts

@@ -16,6 +16,13 @@ export type { Workspace, ProjectConfig, ProjectsConfig };
 export interface ProviderConfig {
   apiKey?: string;
   baseUrl?: string;
+  /** Cloudflare AI Gateway configuration */
+  cloudflareGateway?: {
+    /** Cloudflare account ID */
+    accountId: string;
+    /** Gateway name (created in Cloudflare dashboard) */
+    gatewayName: string;
+  };
   [key: string]: unknown;
 }
 
@@ -465,6 +472,24 @@ export class Config {
 //     "baseUrl": "http://localhost:11434/api"  // Optional - only needed for remote/custom URL
 //   }
 // }
+//
+// Cloudflare AI Gateway example:
+// {
+//   "anthropic": {
+//     "apiKey": "sk-ant-...",
+//     "cloudflareGateway": {
+//       "accountId": "your-cloudflare-account-id",
+//       "gatewayName": "your-gateway-name"
+//     }
+//   },
+//   "openai": {
+//     "apiKey": "sk-...",
+//     "cloudflareGateway": {
+//       "accountId": "your-cloudflare-account-id",
+//       "gatewayName": "your-gateway-name"
+//     }
+//   }
+// }
 ${jsonString}`;
 
       fs.writeFileSync(this.providersFile, contentWithComments);

src/node/services/aiService.ts

@@ -132,6 +132,21 @@ function parseModelString(modelString: string): [string, string] {
   return [providerName, modelId];
 }
 
+/**
+ * Construct Cloudflare AI Gateway URL for a provider
+ * @param accountId - Cloudflare account ID
+ * @param gatewayName - Gateway name from Cloudflare dashboard
+ * @param provider - Provider name (openai, anthropic, google, etc.)
+ * @returns Gateway URL for the provider
+ */
+function buildCloudflareGatewayURL(
+  accountId: string,
+  gatewayName: string,
+  provider: string
+): string {
+  return `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayName}/${provider}`;
+}
+
 export class AIService extends EventEmitter {
   private readonly streamManager: StreamManager;
   private readonly historyService: HistoryService;
@@ -280,10 +295,34 @@ export class AIService extends EventEmitter {
       let providerConfig = providersConfig?.[providerName] ?? {};
 
       // Map baseUrl to baseURL if present (SDK expects baseURL)
-      const { baseUrl, ...configWithoutBaseUrl } = providerConfig;
-      providerConfig = baseUrl
-        ? { ...configWithoutBaseUrl, baseURL: baseUrl }
-        : configWithoutBaseUrl;
+      const { baseUrl, cloudflareGateway, ...configWithoutBaseUrl } = providerConfig;
+
+      // If Cloudflare AI Gateway is configured, use it as baseURL (takes precedence over baseUrl)
+      if (cloudflareGateway && typeof cloudflareGateway === "object") {
+        const { accountId, gatewayName } = cloudflareGateway as {
+          accountId?: string;
+          gatewayName?: string;
+        };
+        if (accountId && gatewayName) {
+          providerConfig = {
+            ...configWithoutBaseUrl,
+            baseURL: buildCloudflareGatewayURL(accountId, gatewayName, providerName),
+          };
+          log.info("Using Cloudflare AI Gateway", {
+            provider: providerName,
+            accountId,
+            gatewayName,
+          });
+        } else {
+          providerConfig = baseUrl
+            ? { ...configWithoutBaseUrl, baseURL: baseUrl }
+            : configWithoutBaseUrl;
+        }
+      } else {
+        providerConfig = baseUrl
+          ? { ...configWithoutBaseUrl, baseURL: baseUrl }
+          : configWithoutBaseUrl;
+      }
 
       // Handle Anthropic provider
       if (providerName === "anthropic") {
@@ -350,7 +389,10 @@ export class AIService extends EventEmitter {
               })();
 
               const method = (init?.method ?? "GET").toUpperCase();
-              const isOpenAIResponses = /\/v1\/responses(\?|$)/.test(urlString);
+              // Match /responses anywhere in path (supports both direct and gateway URLs)
+              // Direct: /v1/responses
+              // Gateway: /v1/{account}/{gateway}/openai/responses
+              const isOpenAIResponses = /\/responses(\?|$)/.test(urlString);
 
               const body = init?.body;
               if (

src/node/services/workspaceTitleGenerator.ts

@@ -1,11 +1,45 @@
 import { generateObject, type LanguageModel } from "ai";
 import { z } from "zod";
-import type { Config } from "@/node/config";
+import type { Config, ProviderConfig } from "@/node/config";
 import { log } from "./log";
 import { createAnthropic } from "@ai-sdk/anthropic";
 import { createOpenAI } from "@ai-sdk/openai";
 import { MODEL_NAMES } from "@/common/constants/knownModels";
 
+/**
+ * Build configuration with Cloudflare AI Gateway support
+ * @param providerConfig - Provider configuration from providers.jsonc
+ * @param providerName - Provider name (openai, anthropic, etc.)
+ * @returns Configuration object with baseURL set if gateway is configured
+ */
+function buildProviderConfig(
+  providerConfig: ProviderConfig,
+  providerName: string
+): Record<string, unknown> {
+  const { cloudflareGateway, baseUrl, ...rest } = providerConfig;
+
+  // If Cloudflare AI Gateway is configured, use it as baseURL
+  if (cloudflareGateway && typeof cloudflareGateway === "object") {
+    const { accountId, gatewayName } = cloudflareGateway as {
+      accountId?: string;
+      gatewayName?: string;
+    };
+    if (accountId && gatewayName) {
+      return {
+        ...rest,
+        baseURL: `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayName}/${providerName}`,
+      };
+    }
+  }
+
+  // Otherwise use baseUrl if provided (mapped to baseURL)
+  if (baseUrl) {
+    return { ...rest, baseURL: baseUrl };
+  }
+
+  return rest;
+}
+
 const workspaceNameSchema = z.object({
   name: z
     .string()
@@ -70,17 +104,15 @@ function getModelForTitleGeneration(modelString: string, config: Config): Langua
   try {
     // Try Anthropic Haiku first (fastest/cheapest)
     if (providersConfig.anthropic?.apiKey) {
-      const provider = createAnthropic({
-        apiKey: String(providersConfig.anthropic.apiKey),
-      });
+      const config = buildProviderConfig(providersConfig.anthropic, "anthropic");
+      const provider = createAnthropic(config);
       return provider(MODEL_NAMES.anthropic.HAIKU);
     }
 
     // Try OpenAI GPT-5-mini second
     if (providersConfig.openai?.apiKey) {
-      const provider = createOpenAI({
-        apiKey: String(providersConfig.openai.apiKey),
-      });
+      const config = buildProviderConfig(providersConfig.openai, "openai");
+      const provider = createOpenAI(config);
       return provider(MODEL_NAMES.openai.GPT_MINI);
     }
 
@@ -92,16 +124,14 @@ function getModelForTitleGeneration(modelString: string, config: Config): Langua
     }
 
     if (providerName === "anthropic" && providersConfig.anthropic?.apiKey) {
-      const provider = createAnthropic({
-        apiKey: String(providersConfig.anthropic.apiKey),
-      });
+      const config = buildProviderConfig(providersConfig.anthropic, "anthropic");
+      const provider = createAnthropic(config);
       return provider(modelId);
     }
 
     if (providerName === "openai" && providersConfig.openai?.apiKey) {
-      const provider = createOpenAI({
-        apiKey: String(providersConfig.openai.apiKey),
-      });
+      const config = buildProviderConfig(providersConfig.openai, "openai");
+      const provider = createOpenAI(config);
       return provider(modelId);
     }