Configuration

Complete reference for AI Proxy configuration options.

Related:

• Getting Started - Basic setup guide
• Monitoring - Health checks and stats endpoints
• Security Configuration - Admin authentication and secure deployment
• Z.ai Knowledge Base - Provider-specific configuration

New to configuration? Start with the Getting Started guide for basic setup. You only need to read this if you want to customize advanced settings.

What Are Environment Variables?

Environment variables are settings that control how the proxy behaves. Think of them like configuration knobs you can adjust without changing any code.

How to set them:

Mac/Linux:

export GLM_PORT=8080
npm start

Windows PowerShell:

$env:GLM_PORT="8080"
npm start

Windows Command Prompt:

set GLM_PORT=8080 && npm start

Quick Reference: For the most commonly used environment variables, see the root README.md.

Environment Variables

Additional Environment Variables

Security Settings: See Security Configuration for admin authentication and security modes.

See the README.md for the full list of additional environment variables including weighted selection, cost tracking, admin auth, and tier downgrade settings.

Understanding Key Concepts

Before configuring advanced settings, it helps to understand these concepts:

What is “Concurrency”?

Concurrency means “how many things happening at the same time.” If you set GLM_MAX_CONCURRENCY_PER_KEY=5, it means the proxy will send at most 5 simultaneous requests to each API key. Any additional requests will wait in a queue.

What is a “Rate Limit”?

A rate limit is like a speed limit for API requests. It says “you can only make X requests per minute.” If you go faster, the API will say “slow down!” (HTTP 429 error). The proxy helps you stay under this limit by spreading requests across multiple keys.

What is a “Circuit Breaker”?

Imagine a safety switch in your house that turns off power when there’s an electrical problem. A circuit breaker does the same thing for API keys: if a key keeps failing, the proxy stops using it temporarily to prevent more errors. See the “Circuit Breaker” section below for details.

What is “Upstream” vs “Downstream”?

• Upstream: The API service you’re connecting to (like Z.AI or Anthropic)
• Downstream: Your application that’s making requests
• Proxy: Sits in the middle, managing the connection between them

api-keys.json Format

The api-keys.json file contains your API credentials:

{
  "keys": [
    "key-id-1.secret-1",
    "key-id-2.secret-2"
  ],
  "baseUrl": "https://api.z.ai/api/anthropic"
}

Need API keys? Sign up at z.ai and create API keys in your dashboard. See Z.ai Documentation for tier comparisons and pricing.

Fields

Note: Z.ai uses the format sk-sp-xxxxx for Coding Plan subscriptions. See Z.ai Knowledge Base for endpoint details.

Circuit Breaker

Note: For circuit breaker architecture diagrams, see Architecture.

What is a Circuit Breaker?

A circuit breaker is a safety feature that protects against failing API keys.

Think of it like this: Imagine you have 5 light bulbs in parallel, and one keeps burning out. A circuit breaker detects which bulb is faulty and turns it off, while the other 4 keep working. When the faulty bulb is replaced, the circuit breaker turns it back on.

In practice:

• If an API key starts failing (returns errors), the proxy stops using it
• Other healthy keys continue working normally
• After a cooldown period, the proxy tries the failed key again
• If it works, the key is restored. If it still fails, it stays disabled

You can see circuit breaker states in the dashboard’s System page:

States

Transitions

• CLOSED → OPEN: After GLM_CIRCUIT_THRESHOLD failures within GLM_CIRCUIT_WINDOW ms
• OPEN → HALF_OPEN: After GLM_CIRCUIT_COOLDOWN ms (waiting period)
• HALF_OPEN → CLOSED: Test request succeeds → key is restored
• HALF_OPEN → OPEN: Test request fails → key stays disabled

Rate Limiting

What is Rate Limiting?

Rate limiting controls how many requests you can make in a time period (usually per minute). Think of it like a speed limit on a highway — if you go too fast, you’ll get pulled over (receive a 429 error).

How the Proxy Handles Rate Limits

The proxy uses a token bucket algorithm to manage rate limits. Here’s what that means in simple terms:

• Rate: GLM_RATE_LIMIT requests per minute (set to 0 to disable)
• Burst: Allows short bursts above the rate limit temporarily

Think of it like this: You have a bucket that fills up with “tokens” over time. Each request uses one token. If the bucket is empty, you have to wait for more tokens. But the bucket can hold some extra tokens, allowing short bursts of activity.

Transient Overflow Retry

What is Transient Overflow Retry?

When enabled via GLM_TRANSIENT_OVERFLOW_RETRY=true, the proxy will retry requests that receive 429 responses due to temporary model capacity issues. Instead of immediately returning a 503 Service Unavailable to the client, the proxy applies exponential backoff and retries up to 4 times.

When is this useful?

This is especially helpful for high-throughput deployments where brief capacity spikes are common. Rather than failing requests due to temporary model overload, the proxy can absorb the spike and eventually get the request through.

How it works:

1. Request is sent to upstream API
2. API returns 429 (capacity exceeded)
3. Proxy applies exponential backoff (increasing wait times)
4. Proxy retries up to 4 times
5. If successful, request completes normally
6. If all retries fail, proxy returns 503 to client

Default: Disabled (safe for canary rollout)

Request Queue

What is the Request Queue?

When all API keys are busy handling their maximum number of concurrent requests, new requests don’t fail immediately — they wait in a queue (a waiting line).

Think of it like a line at a store: If all cashiers are busy, customers wait in line. When a cashier becomes free, the next person in line gets served.

What happens when the queue is full or timeout expires?

The client receives:

• Status: 503 Service Unavailable
• Header: Retry-After: N (tells the client how many seconds to wait before trying again)

Running in Production

Note: For production deployment details, see the root README.md.

Single Process Mode

# Mac/Linux
NO_CLUSTER=1 npm start

# Windows PowerShell
$env:NO_CLUSTER="1"; npm start

# Windows Command Prompt
set NO_CLUSTER=1 && npm start

With PM2

# Mac/Linux
pm2 start proxy.js --name glm-proxy

# Windows PowerShell
pm2 start proxy.js --name glm-proxy

With systemd

[Unit]
Description=AI Proxy
After=network.target

[Service]
Type=simple
User=www-data
WorkingDirectory=/opt/glm-proxy
ExecStart=/usr/bin/node proxy.js
Restart=on-failure
Environment=NODE_ENV=production

[Install]
WantedBy=multi-user.target

Hot Reload

Reload API keys without restart:

# Mac/Linux
curl -X POST http://127.0.0.1:18765/reload

# Windows PowerShell
Invoke-WebRequest -Uri http://127.0.0.1:18765/reload -Method POST

See Also

• Getting Started — Step-by-step setup guide
• Monitoring — Health checks and statistics endpoints
• Architecture - Circuit Breaker — Circuit breaker design and diagrams
• README.md — Project overview and quick start
• TROUBLESHOOTING.md — Common issues and solutions