Streaming (SSE)

Low-latency Server-Sent Events over HTTP. Real-time token streaming across OpenAI and Anthropic client SDKs.

Toknix supports real-time token streaming using standard HTTP Server-Sent Events (SSE). When streaming is enabled with "stream": true, responses are flushed incrementally as upstream provider tokens generate.

SSE Protocol Wire Format

The gateway delivers events prefixed with data: and terminated by \n\n. When generation finishes, the stream closes with the sentinel message data: [DONE]:

text
data: {"id":"chatcmpl-91","choices":[{"delta":{"content":"Toknix"}}]}

data: {"id":"chatcmpl-91","choices":[{"delta":{"content":" delivers"}}]}

data: {"id":"chatcmpl-91","choices":[{"delta":{"content":" edge"}}]}

data: {"id":"chatcmpl-91","choices":[{"delta":{"content":" speed."}}]}

data: [DONE]

Python Streaming

python
from openai import OpenAI

client = OpenAI(
    api_key="tk_live_...",
    base_url="https://api.toknix.in/v1"
)

stream = client.chat.completions.create(
    model="gpt-6-astra",
    messages=[{"role": "user", "content": "Write a poem about distributed systems."}],
    stream=True
)

for chunk in stream:
    content = chunk.choices[0].delta.content or ""
    print(content, end="", flush=True)

Node.js / TypeScript Streaming

typescript
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOKNIX_API_KEY,
  baseURL: "https://api.toknix.in/v1",
});

async function run() {
  const stream = await client.chat.completions.create({
    model: "claude-opus-5-5",
    messages: [{ role: "user", content: "Explain vector databases." }],
    stream: true,
  });

  for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0]?.delta?.content || "");
  }
}

run();

cURL Terminal Streaming

Use the -N (no buffer) flag in cURL to stream tokens directly into your terminal window:

bash
curl -N https://api.toknix.in/v1/chat/completions   -H "Authorization: Bearer $TOKNIX_API_KEY"   -H "Content-Type: application/json"   -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Count from 1 to 10 slowly."}],
    "stream": true
  }'