GPT-Load

English | 中文 | 日本語

GPT-Load is a self-hosted AI API key aggregator and native-protocol gateway written in Go. A single binary with an embedded admin UI manages keys for OpenAI, Anthropic, Gemini, and compatible upstreams while exposing each provider's native data-plane endpoints.

For the maintained 1.4.x release documentation, visit the official documentation.

Sponsors

	Thanks to TeamoRouter for sponsoring this project! TeamoRouter is an enterprise-grade Agentic LLM gateway that lets developers, AI teams, and businesses access Claude Code, Codex, Gemini CLI, and other AI agents through one unified API without separate subscriptions, with discounts of up to 90%. It connects to official providers and trusted partners like OpenAI, Anthropic, Vertex, Azure, and AWS Bedrock, offering verified Agent protocol compatibility, request traceability, near-official TTFT, 99.6% SLA, and up to 5,000 QPM. It also includes centralized billing, team management, BYOK, smart routing, analytics, provider optimization, and dedicated support. Teamo Desktop enables one-click setup with no API key management or manual configuration, and new users can register via this link for 10% off their first top-up.
	Thanks to Unity2.ai for sponsoring this project! Unity2.ai is a high-performance AI model API relay platform for individual developers, teams, and enterprises. It has long served leading enterprises in China, handles over 30 billion token calls per day, and supports 5000 RPM high concurrency. It supports balance billing, first top-up bonuses, bundled subscriptions, enterprise invoicing, and dedicated integration support. Register via this link to receive a $2 balance; join the official group for another $10 balance, up to $12 in free credits.
	Thank you very much for the support from the LINUX DO community!
	This project is supported by DigitalOcean.

2.0 release status

Warning

2.0 is currently in release-ready closeout. This does not mean that a v2.0.0 tag, GitHub Release, binary, or container image has been published. Check the actual release assets before deploying; a repository branch is not evidence of a successful release.

2.0 is a greenfield rewrite whose data is incompatible with 1.x. main remains the 1.4.x maintenance line. 2.0 does not automatically move latest; stable container channels use explicit 2, 2.0, and v2.0.0 tags.

2.0 capabilities

• Two planes: provider-native paths on the data plane; management APIs under /api, with the admin UI embedded in the same Go binary.
• Three native dialects: OpenAI, Anthropic, and Gemini requests are forwarded in their respective protocols. GPT-Load does not translate between protocols.
• Key and traffic management: Groups, encrypted upstream keys, AccessKeys, model discovery, filtering and rate limits, scheduling, health state, cooldown, blacklist, and automatic weights.
• Control and observability: runtime settings, route inspection, health views, RequestLog, and a Chinese, English, and Japanese admin UI.
• Usage and estimated cost: usage extraction for the three dialects, 24-hour/30-day reports, per-request quality states, built-in prices, and user price overrides.

Prices and costs are best-effort estimates derived from upstream usage and the active pricing rules. They are not a billing ledger, invoice, or provider bill, and historical requests are not repriced.

2.0.0 support boundaries

• Correctness is guaranteed for a single application instance only; multi-instance coordination is not supported.
• SQLite only; PostgreSQL, MySQL, and other databases are not supported.
• The AccessKey and runtime configuration select the Group. A Group never appears in the data-plane URL.
• Upstream keys must be encrypted at rest with no plaintext fallback. 2.0.0 has no master-key rotation; migrate-keys remains an explicitly failing deferred command.
• There is no automatic 1.x migration, in-place upgrade, or reverse synchronization.
• There is no protocol conversion, online billing reconciliation, automatic price fetcher, online backup API, or backup CLI.

Quick start

Docker Compose

The 2.x Compose release contract uses ghcr.io/tbphp/gpt-load:2, container path /app/data, and the gpt-load-data named volume. It never uses latest. Check the current checkout first:

cp .env.example .env
docker compose config

Continue only if the resolved configuration uses image ghcr.io/tbphp/gpt-load:2, sets DATA_DIR=/app/data and DATABASE_DSN=/app/data/gpt-load.db, and mounts a named volume at /app/data. If it resolves to latest or a host bind mount, do not use that Compose file for a 2.0 production deployment, and do not substitute latest.

After those preconditions are met:

docker compose up -d
curl --fail http://localhost:3001/health
# If AUTH_KEY was generated on first boot, read it once in a secure terminal
# and immediately store it in a secret manager.
docker compose exec gpt-load sh -c 'cat /app/data/auth.key'

The named volume preserves SQLite, auth.key, and encryption.key. Production deployments should inject explicit AUTH_KEY and ENCRYPTION_KEY values through protected secret handling. Never commit real secrets to .env, logs, or issues. A custom container DATABASE_DSN requires a Compose override with both a container path and a matching volume mount.

Native binary

After publication, download the platform-matching artifact from the GitHub Release and verify it against SHA256SUMS. Until release assets actually exist, build from the current checkout as shown under “Build and verification”; do not assume that an artifact has been published.

Linux amd64 example:

chmod +x ./gpt-load-linux-amd64
mkdir -p ./data
DATA_DIR=./data ./gpt-load-linux-amd64

In another terminal:

curl --fail http://localhost:3001/health

Then open http://localhost:3001 in a browser.

AUTH_KEY and ENCRYPTION_KEY may both be supplied explicitly. When left empty, first boot creates and reuses ${DATA_DIR}/auth.key and ${DATA_DIR}/encryption.key, respectively. The application logs generated file paths, never their secret contents.

Native data plane

Data-plane requests use an AccessKey. Provider-compatible credentials are accepted through Authorization: Bearer, x-api-key, x-goog-api-key, or Gemini's key query parameter as appropriate.

Provider	Method and path	Behavior
OpenAI	POST /v1/chat/completions	Native OpenAI Chat Completions request
OpenAI / Anthropic	GET /v1/models	OpenAI shape by default; Anthropic shape when anthropic-version is present
Anthropic	POST /v1/messages	Native Anthropic Messages request
Gemini	GET /v1beta/models	Native Gemini model list
Gemini	POST /v1beta/models/{model}:generateContent	Gemini non-streaming generation
Gemini	POST /v1beta/models/{model}:streamGenerateContent	Gemini streaming generation

GPT-Load does not translate one dialect into another. The AccessKey and runtime configuration select the Group; it is not passed as a URL path segment.

Management, usage, and cost

The admin UI is served at /, and management APIs are under /api; both use AUTH_KEY. The UI covers Groups, upstream keys, AccessKeys, runtime settings, health, logs, route inspection, Usage, and model-price management. Current code and UI are the management API reference; this README intentionally avoids copying a route list that can drift.

Usage/Cost quality boundaries:

• complete + priced requests contribute to default token and estimated-cost totals.
• missing, partial, and unpriced requests still contribute to request and quality counts but not to default token/cost totals. complete + unpriced requests are never assigned guessed prices.
• A clean EOF on a stream does not guarantee complete usage, and compatible relays may omit the provider's official terminal usage.
• Price changes affect future writes only. Historical RequestLog and UsageStat rows are not recalculated.
• Current-process dropped/write-failure counters and durable database-window aggregates have different scopes.

Core configuration

Variable	Default	Purpose
HOST	0.0.0.0	HTTP listen address
PORT	3001	HTTP listen port
DATA_DIR	./data	Persistent directory for a native process; fixed to /app/data by the container release contract
DATABASE_DSN	${DATA_DIR}/gpt-load.db	SQLite path/DSN; a container path must exist in the container namespace and have a matching volume
AUTH_KEY	generated keyfile	Management bearer credential; an explicit value cannot contain whitespace; otherwise reads or creates ${DATA_DIR}/auth.key
ENCRYPTION_KEY	generated keyfile	Master key for encrypted upstream keys; otherwise reads or creates ${DATA_DIR}/encryption.key
GRACEFUL_SHUTDOWN_TIMEOUT	10	Graceful shutdown timeout in seconds
READ_TIMEOUT	60	Maximum time to read a complete request, in seconds
IDLE_TIMEOUT	120	Keep-alive idle timeout in seconds
CONTAINER_STOP_GRACE_PERIOD	15s	Compose stop budget; must exceed the application shutdown timeout
LOG_LEVEL	info	Application log level
LOG_FORMAT	text	Log format: text or json

See .env.example for the complete process configuration. Connect, first-byte, request, and stream-idle timeouts plus RequestLog retention are runtime settings managed through the admin UI/API, not additional environment variables.

Persistence and security

• By default, ${DATA_DIR} contains SQLite, auth.key, and encryption.key. Protect and back up these assets as one recovery set.
• Losing or replacing encryption.key makes encrypted upstream keys unreadable. 2.0.0 has no automatic repair or master-key rotation.
• An external DATABASE_DSN or explicitly managed secrets must be backed up separately; backing up DATA_DIR no longer covers those external assets.
• SQLite uses WAL. Before backup, stop incoming traffic, send SIGTERM, wait for a clean process exit, and then copy the full persistent asset set. Never copy only gpt-load.db while the service is running.
• Never paste AUTH_KEY, ENCRYPTION_KEY, AccessKeys, or upstream keys into logs, public issues, screenshots, or ordinary backup manifests.

Public operations baseline

This checklist is self-contained and does not require access to the project's private Notion workspace:

1. Before backup or cutover, call authenticated GET /api/system/info. Record the resolved data_dir, database location, and secret sources without recording secret values.
2. Stop incoming traffic, send SIGTERM, and wait for a clean process exit. With Compose, run docker compose stop and confirm the service container is stopped.
3. Archive the complete resolved DATA_DIR or exact named volume as one recovery set. Use a unique archive name, refuse overwrite, restrict access, and record its SHA-256. Back up any external DATABASE_DSN, AUTH_KEY, and ENCRYPTION_KEY separately.
4. Restore with the exact same binary or image into an empty target. Verify the checksum first and restore the matching encryption key; never combine restore with an upgrade.
5. Start the restored instance and verify /health, /api/system/info, Groups, AccessKeys, model prices, Usage, RequestLog, and a real data-plane canary. When sqlite3 is available, stop the instance and require PRAGMA quick_check to return ok.

2.0.0 has no backup CLI or encryption-key rotation. Never replace the encryption key for an existing database.

Moving from 1.x

2.0 cannot open, import, or upgrade a 1.x database in place, and it must not reuse a 1.x DATA_DIR. The recommended flow is:

1. Keep 1.x running and verify that its backup can be restored.
2. Give 2.0 a separate port, DATA_DIR / named volume, and database.
3. Manually rebuild the minimum Groups, upstream keys, AccessKeys, and rules; validate all three dialects, logs, and usage/cost in isolation.
4. Move entry traffic during a maintenance window or small rollout. On failure, stop 2.0 and switch back to the original 1.x deployment; do not reverse-import new 2.0 data.

latest is not a safe 1.x-to-2.0 upgrade channel. Use the public operations baseline above for backup and restore, and keep the original 1.x deployment and data intact until the rollback window closes.

Build and verification

Baseline tools: Go 1.25.12, Node.js >=24.11.0, and pnpm 11.15.1.

Build the single binary with its embedded admin UI:

make build

Full local quality gates:

corepack pnpm --dir web install --frozen-lockfile
corepack pnpm --dir web run lint
corepack pnpm --dir web run format
corepack pnpm --dir web run type-check
corepack pnpm --dir web run test
corepack pnpm --dir web run build
go build -o gpt-load .
go test -race . ./internal/...
corepack pnpm --dir web run test:e2e

2.0.0 is expected to provide five native raw binaries plus SHA256SUMS:

• gpt-load-linux-amd64
• gpt-load-linux-arm64
• gpt-load-macos-amd64
• gpt-load-macos-arm64
• gpt-load-windows-amd64.exe

These are the expected names in the release contract, not a claim that a downloadable GitHub Release already exists.

License and security

GPT-Load is released under the MIT License. Report vulnerabilities through the process in SECURITY.md.