AI Gateway API

One OpenAI-compatible endpoint in front of every model you connect: OpenAI, Anthropic, Google Gemini, Cloudflare Workers AI, or any OpenAI-compatible server.

How to use the API

Four steps from zero to a working integration — everything speaks standard OpenAI, so any existing SDK or HTTP client fits:

1. Get a key. Ask the gateway administrator for an sk- key (dashboard → API Keys — shown once when created). Send it on every call as Authorization: Bearer sk-YOUR_KEY. Keys can be restricted to certain models, a requests-per-minute limit, a monthly request quota, and an expiry date.
2. Discover what to call. GET /v1/models returns everything your key may use: concrete provider models, custom gateway aliases, and the built-in aliases auto, fast, smart, balance. Prefer aliases — they route across providers at call time, so your app never hard-codes a model id that may disappear or change price.
3. Send a chat completion. POST /v1/chat/completions with the standard OpenAI body — only messages is required (or the gateway has a default model). With an OpenAI SDK, set baseURL to https://llm.kaihong.my/v1 and pass your key as-is. Copy-paste curl / Python / JavaScript / Go examples in Quick Start.
4. Read responses and errors. Success is a normal OpenAI chat.completion JSON (choices, usage); add stream: true for server-sent events. Failures return {"error": {"message": "...", "type": "..."}} — branch on type (see the table in Errors) and retry 429.

The complete client surface — three endpoints:

Authentication

Every call to /v1/* requires an API key issued by the gateway administrator. Send it as a bearer token (or via the x-api-key header).

Authorization: Bearer sk-YOUR_KEY

Keys can be restricted to specific models and rate-limited (requests per minute). A revoked or expired key returns 401.

Quick Start

Send a chat completion with any alias or concrete model id:

curl https://llm.kaihong.my/v1/chat/completions -H "Authorization: Bearer sk-YOUR_KEY" -H "Content-Type: application/json" -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

import requests

url = "https://llm.kaihong.my/v1/chat/completions"
headers = {"Authorization": "Bearer sk-YOUR_KEY"}
payload = {
    "model": "auto",
    "messages": [{"role": "user", "content": "Hello!"}],
}

r = requests.post(url, headers=headers, json=payload)
print(r.json()["choices"][0]["message"]["content"])

const res = await fetch("https://llm.kaihong.my/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer sk-YOUR_KEY",
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: "auto",
    messages: [{ role: "user", content: "Hello!" }]
  })
});

const data = await res.json();
console.log(data.choices[0].message.content);

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
)

func main() {
    body, _ := json.Marshal(map[string]interface{}{
        "model": "auto",
        "messages": []map[string]string{
            {"role": "user", "content": "Hello!"},
        },
    })
    req, _ := http.NewRequest("POST", "https://llm.kaihong.my/v1/chat/completions", bytes.NewReader(body))
    req.Header.Set("Authorization", "Bearer sk-YOUR_KEY")
    req.Header.Set("Content-Type", "application/json")
    res, err := http.DefaultClient.Do(req)
    if err != nil {
        panic(err)
    }
    defer res.Body.Close()
    out, _ := io.ReadAll(res.Body)
    fmt.Println(string(out))
}

Aliases

Aliases are virtual model names that always resolve to the best provider/model at call time. Use them instead of hard-coding model ids and your app keeps working across providers, price changes, and outages.

These four names are reserved — they can never be used as a gateway model alias. Define additional virtual model names as gateway models in the dashboard (Gateway Models page): each is defined by its aliases — the names you call (at least one required; gateway models have no id and can be enabled/disabled without deleting them), an ordered target list, fallback aliases and default params.

Chat Completions

POST /v1/chat/completions — standard OpenAI request body; only messages is required. Any OpenAI body field is accepted: OpenAI-compatible providers receive it unchanged (including tools and response_format), while translated providers (Anthropic, Gemini, Ollama, Cohere, Bedrock) map the fields they support — at least model, messages, stream, temperature, top_p, max_tokens and stop.

{
  "model": "auto",
  "messages": [
    { "role": "system", "content": "You are concise." },
    { "role": "user", "content": "Summarize quantum computing in two sentences." }
  ],
  "temperature": 0.7,
  "max_tokens": 200
}

Non-streaming responses carry three debug headers — x-gateway-provider, x-gateway-model and x-gateway-attempts — showing which provider actually served the call.

Streaming

Set stream: true to receive server-sent events in OpenAI chunk format, terminated by data: [DONE].

const res = await fetch("https://llm.kaihong.my/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer sk-YOUR_KEY",
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: "auto",
    stream: true,
    messages: [{ role: "user", content: "Tell me a short story." }]
  })
});

const reader = res.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  const lines = decoder.decode(value).split(String.fromCharCode(10));
  for (const line of lines) {
    if (line.indexOf("data:") !== 0) continue;
    const payload = line.slice(5).trim();
    if (payload === "[DONE]") continue;
    const chunk = JSON.parse(payload);
    const delta = chunk.choices[0].delta.content;
    if (delta) process.stdout.write(delta);
  }
}

List Models

GET /v1/models — returns every enabled concrete model plus the virtual entries: gateway-model aliases, routes, and the built-in aliases. Gateway models have no id — every enabled alias is listed as its own entry, with alias_of naming the gateway model's first alias; disabled gateway models are not listed.

curl https://llm.kaihong.my/v1/models -H "Authorization: Bearer sk-YOUR_KEY"

{
  "object": "list",
  "data": [
    {
      "id": "gpt-4o-mini",
      "object": "model",
      "provider": "OpenAI",
      "latency_ms": 420
    },
    {
      "id": "best",
      "object": "model",
      "gateway": true,
      "alias_of": "premium",
      "targets": ["OpenAI/gpt-4o-mini"]
    },
    {
      "id": "auto",
      "object": "model",
      "alias": true,
      "resolves_to": { "provider_id": "...", "model_id": "gpt-4o-mini" },
      "provider": "OpenAI",
      "latency_ms": 420
    }
  ]
}

Errors

Errors use a consistent JSON envelope. The type field is stable and safe to branch on.

{
  "error": {
    "message": "Model "xyz" is not allowed for this API key",
    "type": "model_not_allowed"
  }
}

AI Gateway · OpenAI-compatible API