GPT-Load

A self-hosted AI gateway for multi-channel, multi-credential setups

API keys, subscription accounts, traffic scheduling, failure handling, request logs, and usage accounting — behind a single entry point.

English · 中文 · 日本語

---

Your application only needs one base URL and one AccessKey. Providers, accounts, credentials, models, and routing policy are all configured in the management UI.

flowchart LR
    C["Application / AI client"]
    G["GPT-Load<br/>————————<br/>Native protocol entry<br/>Scheduling · Retry · Health isolation<br/>Logs · Usage · Cost estimates"]
    U1["Official APIs"]
    U2["Cloud platforms"]
    U3["Compatible relays"]
    U4["Subscription accounts"]

    C -->|"One base URL<br/>One AccessKey"| G
    G --> U1
    G --> U2
    G --> U3
    G --> U4

Why GPT-Load

• One gateway for many upstreams — Official APIs, cloud platforms, popular model services, and OpenAI-compatible relays are all managed in one place.
• API keys and subscription accounts share one mechanism — Subscription channels like Codex, Claude, Antigravity, and Grok use the same credential management, scheduling, and health system as API key channels.
• Get the most out of a credential pool — Multi-credential scheduling, automatic weighting, retries, cooldown, blacklisting, and session affinity reduce the impact of a single overloaded or failing credential.
• Clients keep their native protocol — Applications keep using OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, or the Gemini native API without code changes.
• Every call is visible — Health state, route inspection, request logs, usage rollups, and per-model cost estimates make problems and consumption easy to trace.
• Simple to deploy, your data stays yours — The management UI is embedded in a single Go binary; SQLite by default, MySQL or PostgreSQL optional; channel credentials are encrypted at rest locally.

Screenshots

Home — Groups and credentials at a glance, one-click client setup, 30-day cost estimate

Monitoring — Request volume, cache rate, token breakdown, cost estimates, usage quality

Scope

Client protocols

Protocol	Main entry
OpenAI Chat Completions	POST /v1/chat/completions
OpenAI Responses	/v1/responses and its resource paths
Anthropic Messages	POST /v1/messages
Gemini	/v1beta/models/...

Each channel declares exactly which protocols and capabilities it can execute. GPT-Load converts between supported capabilities, but it is not a general-purpose any-protocol, any-JSON translator.

Built-in channels

Show all 20 built-in channels

• Official and cloud — OpenAI, Anthropic, Gemini, xAI, Azure OpenAI, AWS Bedrock, Google Vertex AI
• Model services — DeepSeek, Moonshot AI, SiliconFlow, Zhipu AI, Alibaba, Volcengine, OpenRouter, Groq
• Subscription — Codex, Claude, Antigravity, Grok
• Custom — OpenAI Compatible (any compatible relay)

Quick start

1. Start the service

Requires Docker and Docker Compose.

git clone --depth 1 --branch v2 https://github.com/tbphp/gpt-load.git
cd gpt-load

cp .env.example .env
docker compose up -d

Confirm the service is up:

curl --fail http://127.0.0.1:3001/health

The first start generates a management key. Read it and store it safely:

docker compose exec gpt-load sh -c 'cat /app/data/auth.key'

Open http://127.0.0.1:3001 and sign in to the console with that key.

You can also set AUTH_KEY explicitly in .env before starting. By default the service listens on the loopback address only and is not exposed to the internet.

2. Initial configuration

The console has three configuration layers:

flowchart LR
    A["① Channel<br/>Add API keys<br/>or complete OAuth"] --> B["② Group<br/>Pick a channel<br/>Set models and policy"] --> C["③ AccessKey<br/>Pick groups and protocols<br/>Hand it to your app"]

1. Add a channel — Choose an upstream service and add one or more API keys. For subscription channels, complete the OAuth flow or import credentials as prompted.
2. Create a group — Pick a channel, then configure available models and runtime policy.
3. Create an AccessKey — Set the groups and client protocols it may use, then give the generated AccessKey to your application.

OAuth callback ports for subscription channels

The Codex, Claude, and Antigravity OAuth clients use fixed callback ports. Compose publishes them on the address configured by HOST, which defaults to 127.0.0.1; setting HOST=0.0.0.0 also publishes these callback ports on all host interfaces. Because the ports are fixed by the upstream clients, only one default Compose instance can run on a host at a time.

When working over SSH or from a remote browser, the browser's localhost may not reach GPT-Load — paste the full callback URL into the authorization dialog to finish the flow.

3. Send your first request

Replace the AccessKey and model ID with the real values from your console:

export GPT_LOAD_ACCESS_KEY="your-access-key"

curl http://127.0.0.1:3001/v1/chat/completions \
  -H "Authorization: Bearer ${GPT_LOAD_ACCESS_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "messages": [{ "role": "user", "content": "Hello, introduce yourself." }]
  }'

Using existing clients

For the OpenAI SDK or any OpenAI-compatible client, two settings usually change:

Base URL: http://127.0.0.1:3001/v1
API Key:  the AccessKey created in GPT-Load

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:3001/v1",
    api_key="your-access-key",
)

response = client.responses.create(
    model="your-model-id",
    input="Hello",
    store=False,
)

print(response.output_text)

Anthropic clients use /v1/messages and Gemini clients use /v1beta/models/.... Authenticate the way each client normally does: Authorization: Bearer, x-api-key, x-goog-api-key, or Gemini's key query parameter.

Deployment and data

Docker Compose uses application-managed SQLite by default. Data lives in the gpt-load-data named volume and includes the database, auth.key, and encryption.key.

Important

encryption.key decrypts channel credentials. When backing up or migrating, the database and the key must be kept together. Once the key is lost or replaced, existing encrypted credentials cannot be recovered, and this version does not support master key rotation.

For an external database, use the unified DATABASE_DSN to connect SQLite, MySQL, or PostgreSQL:

mysql://user:password@db.example:3306/gpt_load?charset=utf8mb4&collation=utf8mb4_bin
postgres://user:password@db.example:5432/gpt_load?sslmode=require

See .env.example for the full configuration reference.

Common operations:

docker compose logs -f      # view logs
docker compose pull && docker compose up -d   # update to the latest 2.x image
docker compose stop         # stop the service

The official 2.x Compose file uses ghcr.io/tbphp/gpt-load:v2beta and does not rely on the latest tag.

Using a native binary

Download the build for your platform from GitHub Releases, and verify it against the bundled SHA256SUMS first:

chmod +x ./gpt-load-linux-amd64
mkdir -p ./data

HOST=127.0.0.1 DATA_DIR=./data ./gpt-load-linux-amd64

Then open http://127.0.0.1:3001. Builds are provided for five targets across Linux, macOS (amd64 / arm64), and Windows.

Before you go to production

• The service listens on 127.0.0.1 only by default. For remote access, expose it through a controlled network or a TLS reverse proxy, and configure ACLs and firewall rules.
• Manage AUTH_KEY and ENCRYPTION_KEY carefully. Never commit real keys to a repository, log, screenshot, or public issue.
• 2.0 is designed for a single application instance. Instances do not share state, so horizontal scaling is not supported.
• Usage and cost are estimates derived from upstream responses. They support operational analysis and capacity planning, and do not equal a provider invoice or a financial reconciliation.
• Subscription channels depend on upstream OAuth and compatibility protocols and may change as upstreams change. Only connect accounts you are entitled to use, and follow each provider's terms.
• In OpenAI Responses, stateful requests relying on previous_response_id, conversation, or an existing resource ID are only reliable with a single credential, or with an upstream that shares resources across credentials.

Moving from 1.x

Warning

GPT-Load 2.0 is a complete rewrite. It cannot open, import, or migrate 1.x data in place.

Deploy 2.0 with its own database, DATA_DIR, port, and Docker volume. Cut traffic over only after verification, and keep the original 1.x deployment until the rollback window closes. Documentation for the 1.4.x maintenance line is at the official docs.

Open-source dependencies

Some of GPT-Load's capabilities build on these projects, with thanks:

Project	Role	License
Bifrost Core	Provider authentication, request/response conversion, streaming, usage normalization	Apache-2.0
CLIProxyAPI	OAuth and execution adapter for subscription channels	MIT
Lobe Icons	Channel brand icons in the management UI	MIT

GPT-Load owns credential storage, account selection, scheduling, retry, health, affinity, logging, and usage policy. Third-party notices are in THIRD_PARTY_NOTICES.md, full license texts in LICENSES/, and each release ships a CycloneDX SBOM covering the Go dependency graph.

Channel icons identify their respective upstream providers. All trademarks belong to their owners; this project is not affiliated with or endorsed by them.

Feedback and contributing

For problems or feature ideas, open a GitHub Issue. Report security vulnerabilities through the process in SECURITY.md.

For community chat and usage discussion, join the Telegram group.

If GPT-Load is useful to you, a star is appreciated.

Sponsors and support

	Thanks to APIMart for sponsoring this project! APIMart is a low-cost API platform for AI image & video generation — GPT-Image-2 from $0.006/image, 160+ images per dollar. One async API covers both image and video: submit a task, get an ID, fetch results via polling or callback. Batch tens of thousands of images without timeouts, switch models without changing code. Pay-as-you-go with no monthly fee — sign up here to get started.
	Thanks to OpenAI for sponsoring this project.
	Thanks to the LINUX DO community for their support.
	This project is supported by DigitalOcean.

License

GPT-Load is released under the MIT License.