OpenAI-Compatible API

SwarmLLM provides a drop-in replacement for the OpenAI API. All endpoints require Bearer token authentication.

POST /v1/chat/completions

Chat completions with streaming support.

curl http://localhost:8800/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-coder-7b",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is Rust?"}
    ],
    "stream": true,
    "max_tokens": 512,
    "temperature": 0.7
  }'

Request Body

Response (non-streaming)

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "model": "qwen2.5-coder-7b",
  "choices": [{
    "index": 0,
    "message": {"role": "assistant", "content": "Rust is a systems programming language..."},
    "finish_reason": "stop",
    "logprobs": null
  }],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 42,
    "total_tokens": 57
  }
}

Response with logprobs

When logprobs: true and top_logprobs: 3:

{
  "choices": [{
    "message": {"role": "assistant", "content": "Hello"},
    "finish_reason": "stop",
    "logprobs": {
      "content": [{
        "token": "Hello",
        "logprob": -0.234,
        "bytes": null,
        "top_logprobs": [
          {"token": "Hello", "logprob": -0.234, "bytes": null},
          {"token": "Hi", "logprob": -1.456, "bytes": null},
          {"token": "Hey", "logprob": -2.012, "bytes": null}
        ]
      }]
    }
  }]
}

Response with tool_calls

When the model calls a tool, finish_reason is "tool_calls" and content is null:

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\":\"NYC\"}"
        }
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

Streaming (SSE)

When stream: true, responses arrive as Server-Sent Events:

data: {"id":"chatcmpl-abc123","choices":[{"delta":{"content":"Rust"},"index":0}]}

data: {"id":"chatcmpl-abc123","choices":[{"delta":{"content":" is"},"index":0}]}

data: [DONE]

GET /v1/models

List available models.

curl http://localhost:8800/v1/models \
  -H "Authorization: Bearer YOUR_API_KEY"

{
  "object": "list",
  "data": [
    {
      "id": "qwen2.5-coder-7b-instruct-q4-k-m",
      "object": "model",
      "created": 1756000000,
      "owned_by": "local",
      "max_model_len": 8192,
      "context_length": 8192
    }
  ]
}

owned_by says where the model's parts sit: local (every part is on this computer), hybrid (some parts here, some on peers) or network. It does not say where a request runs. A local model that this computer's graphics card cannot hold whole may still be run with peers' help, and peers then compute on the model's working data for the layers they run. They are not handed the prompt as text, but that data can be turned back into much of it (Security > Start and finish on this computer). To keep one model's requests on this computer, turn on Keep on this computer in the model's ⚙ panel on the Dashboard (or PUT /api/admin/models/{id}/keep-on-this-computer with {"enabled": true}): no other computer, in the swarm or a cloud provider, is then handed anything of those requests, and one this computer cannot run alone is refused with kept_on_this_computer_error (503) instead of being sent out. To keep every request on your own computers, use a private pool with private_mode = true and private_mode_allow_lan = false (Joining the network > Private Networks).

Every reply says which computers answered it. A reply that is not streamed carries the x-swarm-route, x-swarm-segments, x-swarm-peers (how many OTHER computers took part), x-swarm-nodes and, where they declared one, x-swarm-regions headers. The same facts are in the reply itself, as "swarmllm": {"route": {"route": "distributed", "segments": 2, "peers": 1, "nodes": ["0718d8b9", "96842635"], "attempts": 1, "all_nodes": ["0718d8b9", "96842635"], "all_peers": 1}}: in the body of a chat completion or an Anthropic reply, in the last chunk of a streamed chat completion (the one with finish_reason), in a streamed Anthropic reply's message_delta, and in the response object of /v1/responses (also when it is streamed, run in the background or read back later). A streamed reply has no route headers — they are sent before the route exists — so read it there. An MCP tool that asks one model puts it in the result's _meta["com.swarmllm/route"]; compare, research and batch_prompts give each model's result its own route. A reply a cloud provider wrote carries none.

route, segments, peers and nodes describe the computers the reply finished on. When a computer failed or refused part-way and the request was planned again (or its reply carried on elsewhere), the computers it was sent to first saw it too — so all_nodes and all_peers name every computer any attempt reached, this one included, and attempts says how many plans it took (headers x-swarm-all-nodes, x-swarm-all-peers, x-swarm-attempts). A reply that says local with all_peers above 0 was answered here after another computer had been handed it. The dashboard's chat says so beside the route, as "tried 1 other computer first".

To keep a single request on this computer, whatever the model's setting, send the header X-SwarmLLM-Keep-On-This-Computer: true, or put "swarmllm": {"keep_on_this_computer": true} in the body of /v1/chat/completions, /v1/messages or /v1/responses. An MCP tool call takes "_meta": {"com.swarmllm/keep_on_this_computer": true} in its params (or the same header on /mcp), and every model call the tool makes is kept. The request is then treated like one for a kept model: nothing of it goes to another computer or to a cloud provider, and if this computer cannot run it alone it is refused with kept_on_this_computer_error (503). The swarmllm field itself is never passed on to a provider. Whichever instruction is strictest wins — false never undoes the model's own setting — and an instruction the node cannot read (a header value other than true, false, 1, 0, yes or no, or a misspelt field name) is refused with a 400 rather than ignored.

curl http://localhost:8800/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-SwarmLLM-Keep-On-This-Computer: true" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Hello"}]}'

max_model_len and context_length are the same number — the longest conversation this node will serve, after the shipped 8192-token default and any inference.max_seq_len_override. The first is the name vLLM clients read, the second the name OpenClaw's model discovery reads. Both are omitted when the model's declared context cannot be read, rather than guessed. Up to v0.3.224 that is every model no part of which is on this machine; from v0.3.225 such a model reports the context its holders declare, once they run v0.3.225 or later too.

It is THIS node's limit. When a model is split, each computer applies its own, and each advertises it: a prompt is not sent to a computer that has said it serves less, and one that refuses is failed over from. If no computer that could run some part of the model serves the length, the request fails with a 400 that says the limit is set on those computers — raising your own setting does not change it.

GET /v1/status

Node status (SwarmLLM extension).

curl http://localhost:8800/v1/status \
  -H "Authorization: Bearer YOUR_API_KEY"

Using with OpenAI Client Libraries

Python (openai)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8800/v1",
    api_key="YOUR_API_KEY"
)

# Basic streaming
response = client.chat.completions.create(
    model="qwen2.5-coder-7b",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Python — Function calling

response = client.chat.completions.create(
    model="qwen2.5-coder-7b",
    messages=[{"role": "user", "content": "What's the weather in NYC?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather",
            "parameters": {
                "type": "object",
                "properties": {"location": {"type": "string"}},
                "required": ["location"]
            }
        }
    }],
    tool_choice="auto"
)

if response.choices[0].finish_reason == "tool_calls":
    for tc in response.choices[0].message.tool_calls:
        print(f"Call {tc.function.name}({tc.function.arguments})")

JavaScript (openai)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8800/v1",
  apiKey: "YOUR_API_KEY",
});

const stream = await client.chat.completions.create({
  model: "qwen2.5-coder-7b",
  messages: [{ role: "user", content: "Hello!" }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}

curl (streaming)

curl -N http://localhost:8800/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen2.5-coder-7b","messages":[{"role":"user","content":"Hello!"}],"stream":true}'

OpenClaw

OpenClaw accepts any OpenAI-compatible server as a model provider. Export your key as SWARMLLM_API_KEY (the daemon reads the same variable) and add a swarmllm provider to ~/.openclaw/openclaw.json:

{
  models: {
    providers: {
      swarmllm: {
        baseUrl: "http://127.0.0.1:8800/v1",
        apiKey: "${SWARMLLM_API_KEY}",
        api: "openai-completions",
        timeoutSeconds: 300,
        models: [{ id: "meta-llama-3.1-8b-instruct-q4-k-m", name: "SwarmLLM Llama 3.1 8B",
                   contextWindow: 32768, maxTokens: 4096,
                   cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 } }],
      },
    },
  },
  agents: { defaults: { model: { primary: "swarmllm/meta-llama-3.1-8b-instruct-q4-k-m" } } },
}

Two requirements, both on the SwarmLLM side:

• A real key. OpenClaw accepts a non-secret marker such as "ollama-local" for loopback servers, but a marker sends no Authorization header at all, and every /v1 route here requires one.
• A context window above the 8192-token default. OpenClaw's system prompt alone is larger than that, and its context guard refuses a model below 4k tokens and warns below 8k. Set max_seq_len_override = 32768 under [inference] in config.toml (config file only — it is not a dashboard setting or an environment variable).

Model ids are whatever GET /v1/models lists, referenced as swarmllm/<id>. Each entry carries context_length, which is the field OpenClaw's self-hosted discovery reads; with it absent OpenClaw assumes 128,000 and sends prompts the node has to refuse. Embeddings are not served here (see below), so point OpenClaw's memory search at another embedding provider.

POST /v1/embeddings

Returns 501 Not Implemented. Text embeddings are not supported via the subprocess inference path. Use a dedicated embedding provider or the OpenAI embeddings API directly.

GET /v1/providers

List configured cloud providers and their available models.

curl http://localhost:8800/v1/providers \
  -H "Authorization: Bearer YOUR_API_KEY"

Returns an array of { name, models: [...] } objects for each configured provider.