LiteLLM Integration | Tokener CA

Use Cases

LiteLLM can unify different model providers into OpenAI-style calls, and it can also act as an internal LLM Gateway for your team. Tokener CA already provides an OpenAI-compatible /v1/chat/completions, so you can integrate it through LiteLLM's OpenAI provider.

There are two common ways to integrate:

Method	Best for
LiteLLM Python SDK	Calling Tokener CA directly from a Python project
LiteLLM Proxy	Integrating LiteLLM internally first, then letting other OpenAI-compatible clients access LiteLLM

Prepare Environment Variables

export SKYPOOL_API_KEY="stc-..."export SKYPOOL_API_BASE="https://a.skypool.xyz/v1"

PowerShell:

$env:SKYPOOL_API_KEY="stc-..."$env:SKYPOOL_API_BASE="https://a.skypool.xyz/v1"

Method 1: Call Directly from the Python SDK

Install LiteLLM:

uv add litellm

Or use pip:

pip install litellm

Minimal call example:

import osfrom litellm import completion response = completion(    model="openai/gemma4:26b",    api_key=os.environ["SKYPOOL_API_KEY"],    api_base=os.environ.get("SKYPOOL_API_BASE", "https://a.skypool.xyz/v1"),    messages=[        {            "role": "user",            "content": "In one sentence, explain that LiteLLM is already integrated with Tokener CA.",        }    ],    max_tokens=128,) print(response.choices[0].message.content)

Streaming call:

import osfrom litellm import completion stream = completion(    model="openai/gemma4:26b",    api_key=os.environ["SKYPOOL_API_KEY"],    api_base=os.environ.get("SKYPOOL_API_BASE", "https://a.skypool.xyz/v1"),    messages=[        {            "role": "user",            "content": "Introduce Tokener CA in three points.",        }    ],    stream=True,    max_tokens=512,) for chunk in stream:    delta = chunk.choices[0].delta.content    if delta:        print(delta, end="")

Note: LiteLLM's openai/<model> prefix indicates that the call is made in the OpenAI-compatible way; the real model ID is still the Tokener CA platform model name.

Method 2: LiteLLM Proxy

Create litellm_config.yaml:

model_list:  - model_name: "gemma4:26b"    litellm_params:      model: "openai/gemma4:26b"      api_base: os.environ/SKYPOOL_API_BASE      api_key: os.environ/SKYPOOL_API_KEY litellm_settings:  drop_params: true

Start LiteLLM Proxy:

litellm --config litellm_config.yaml --port 4000

Then other OpenAI-compatible clients can access LiteLLM:

from openai import OpenAI client = OpenAI(    api_key="anything",    base_url="http://127.0.0.1:4000",) completion = client.chat.completions.create(    model="gemma4:26b",    messages=[        {            "role": "user",            "content": "Return a one-line message confirming LiteLLM Proxy integration succeeded.",        }    ],) print(completion.choices[0].message.content)

Verify the Upstream

If LiteLLM reports an error, first bypass LiteLLM and verify Tokener CA directly:

curl -X POST "https://a.skypool.xyz/v1/chat/completions" \  -H "Authorization: Bearer <consumer_api_key>" \  -H "Content-Type: application/json" \  --data-raw '{    "model": "gemma4:26b",    "messages": [      {        "role": "user",        "content": "Return a short message confirming the integration succeeded."      }    ],    "max_tokens": 64,    "stream": false  }'

Troubleshooting Checklist

Symptom	How to handle
`AuthenticationError`	Check whether `SKYPOOL_API_KEY` is set, and make sure it is not mixed up with a LiteLLM Proxy virtual key
`NotFoundError`	Check whether `model` is written as `openai/<platform model ID>`
Proxy model not taking effect	Check whether `model_name` matches the `model` in the client request
Incompatible parameters	Keep `drop_params: true` in `litellm_settings`, then enable advanced parameters one by one
Streaming issues	Run a non-streaming request first, then turn on `stream=True`