Skip to main content

​
Multi-provider LLM configuration

Crewship used to reach its LLM providers through a three-arm switch: one case per provider in the aux-slot builder, a second copy in the Keeper validator, and a third inside the error string that told an operator what they were allowed to type. The three drifted the way three copies do — the builder matched "anthropic" case-sensitively while the API carried the uppercase enum, so Anthropic was a working provider on one side of a call and an unsupported one on the other. That switch is now a table (internal/llm/registry.go), and the two hand-written HTTP clients behind it are now two configurable codecs that any compatible backend can be pointed at. What it buys: adding a provider is one registry row plus one price row, and reaching a new OpenAI-compatible backend (DeepSeek, vLLM, llama.cpp, Ollama’s /v1 shim, OpenRouter) is a config value rather than a new file.
Scope. This page is about the server’s own LLM providers — the models Keeper’s judge, the auxiliary evaluators and the in-process call path use. It is not about the container/storage providers at Providers, and it is not about the agent CLI adapters (claude, codex, gemini, cursor-agent, droid) at CLI adapters, which bring their own model configuration and are metered by the sidecar.

​
The provider registry

One row per provider this build can actually construct. Everything that used to be a literal — the picker’s vocabulary, the builder’s switch, the “want anthropic|openai|ollama” hint — reads this table. The three shipped rows, in declaration order: Neither hosted row is endpoint-driven, and that is deliberate: the server’s own key is attached to those requests, so the address they go to has to be ours and not an operator-supplied one. A tenant-chosen endpoint is reached through a vault credential and the SSRF fence instead — see Keeper for the governance model that does exactly that.
Declaration order is load-bearing. It is what the console’s provider picker and crewship keeper aux set --provider render, and keepercfg.AuxProviders() returns it verbatim, so the validator and the builder can no longer disagree about what an operator may type. Sorting the list would silently reorder an operator’s dropdown for no reason anyone could point at.
Provider ids are matched case-insensitively and trimmed. internal/api carries the provider as an uppercase enum, keepercfg stores it lowercase. Anthropic and " anthropic" both resolve now; before the registry they resolved on one side of a call and 400’d on the other.
ID is the paymaster pricing key. A registry row whose ID has no rate row in internal/paymaster/pricing.go and no entry in the embedded catalog bills every call at $0 — silently, with a ledger row that looks fine. Adding a provider is a two-file change by construction. See Pricing: how a provider gets a rate.
Google is absent on purpose. The model catalog carries Gemini ids and GET /api/v1/models will happily list them, but this package has no Gemini Provider implementation — so accepting google on an evaluator slot would give the operator a slot that saves cleanly and then fails at first use. Refusing it with a reason is the honest surface. Registration is init-only: RegisterProvider is called from one func init() in registry.go and the map has no mutex, deliberately, because making registration concurrent-safe would put a lock on a read that sits behind every evaluator call for the benefit of a write that happens once before main. The way to add a provider is a line in that init, not a call from elsewhere.

​
Evaluator defaults follow the credential you actually have

The shipped default for every auxiliary slot is the first registry row that names a DefaultAuxModel — anthropic / claude-haiku-4-5. What an instance boots with is that default retargeted at the first registered provider whose KeyEnv is actually set. An operator holding only an OPENAI_API_KEY used to get six evaluator slots hardcoded to Anthropic, each failing at first use demanding a key they have no reason to own; they now get openai / gpt-5.4-mini. Ties break on declaration order, so an instance holding both keys keeps the shipped Anthropic default — adding a second key never silently repoints a working evaluator. With no credential for any row the slots keep the shipped default and the builder still errors loudly, naming the env var to set. An explicit CREWSHIP_AUX_<SLOT>_PROVIDER always wins over this guess.

​
Codecs

Three wire formats, two of them now parameterized by a config struct whose zero value reproduces the previous hard-coded client byte for byte. That invariant is what let the codecs land without a behaviour change, and it is pinned by tests in internal/llm.

​
OpenAI-compatible (openai-compat)

One implementation, many backends. What is shared is the message model — flat messages, tool_call_id on the tool-result turn, tools:[{type:"function"}], finish_reason — and every backend this codec serves speaks it. What differs is transport and vocabulary, which is what OpenAICompatConfig carries. A backend that differed in the message model would need a second codec, not a config value.
Set IncludeUsage on anything you stream. Without stream_options:{include_usage:true} an OpenAI-compatible backend returns no usage block on a streamed response at all. Every call then reports zero tokens, and paymaster prices zero tokens at $0 — a fleet that looks free and is not. Every shipped preset sets it; a hand-built config must too.
Client is the SSRF seam, and StreamClient carries the caller’s transport and its CheckRedirect. Carrying only the transport would leave Stream following redirects off a fenced endpoint while Complete refuses at the first 3xx — a provider fenced for one call shape and open for the other. Streaming deliberately has no total deadline: http.Client.Timeout covers body read, so a 120s cap silently truncates any generation that runs longer.
The presets this package ships wiring for: ollama-openai and vllm deliberately report a pricing key that is not their preset name: ollama/* and local/* are the two rows in priceTable that are allowed to be free, and a self-hosted backend must land on one of them.
vllm ships no BaseURL, and an unset one becomes api.openai.com. There is no such thing as a default vLLM address, so the preset leaves it empty and withDefaults fills in the hosted OpenAI endpoint — which would send your traffic somewhere you did not ask for. Always set BaseURL after taking the vllm preset.
The preset list is short on purpose. Name doubles as the pricing key, so a preset with no verified price row bills every call at $0. A preset may only be added together with that row, and TestPresetsResolveToAPricedProvider enforces it.

​
Anthropic Messages (anthropic-messages)

Defaults resolve lazily, at each accessor, never at construction — existing tests build a bare &Anthropic{apiKey: "..."} composite literal, so a zero config has to behave like the default one rather than like a half-built object.
There is no Bedrock support. AnthropicConfig declares three fields — Sign, VersionInBody, ModelInPath — that a Bedrock variant would need, and they are honoured where they are read, but no Bedrock preset, no SigV4 code and no AWS dependency ship in this build. The catalog carries amazon-bedrock model ids because it is a verbatim upstream snapshot; that is a price list, not a provider. Read those three fields as a seam, not a feature.

​
Ollama native (ollama-native)

Ollama keeps its own codec because /api/chat is NDJSON rather than SSE and carries the think and format knobs llm.Request exposes — the two switches the Keeper judge depends on to get a parseable verdict out of a small model inside a 256-token budget. The ollama-openai preset is the other way to reach the same daemon, through its /v1 compatibility shim; it speaks the OpenAI message model and therefore does not carry think/format. Rule of thumb: use ollama-native (the ollama registry row) for anything wired into Keeper or an evaluator slot, and ollama-openai when you want the same daemon to stand in for a hosted OpenAI-compatible backend.

​
Walkthrough: a keyless local model

This is the shortest end-to-end proof that the OpenAI-compatible codec works, and it costs nothing and needs no key. Every command below was run as written against a live Ollama daemon, and every output block is what came back — with the box-drawing borders the CLI draws around tables trimmed for width, and with the model’s wording and the exact latency varying between runs, as they will for you.
1

See what this build can talk to

crewship provider list is entirely local — no config file, no token, no reachable server — because the registry and the catalog are both compiled into the binary:
KEY reports only whether the environment variable is set, never its value; not needed is a provider that takes no credential at all. MODELS is how many entries the embedded catalog has for that provider — Ollama’s 0 is correct, since its model ids are whatever you pulled.--all re-prints the same table with the catalog-only providers appended: registry rows first in declaration order, then amazon-bedrock (120), deepseek (4), google (39), mistral (33), openrouter (353) and xai (12), sorted. Every column but PROVIDER, NAME and MODELS is a dash on those rows, and that dash is the whole story: deepseek is listed because the catalog prices it, not because a slot can select it.
2

Start the daemon

3

Pull a small model

A 0.5B model is enough to prove the wiring. It is not a judge — see the model-choice warning in Keeper before pointing anything real at it.
4

Prove the OpenAI-compatible shim answers

The ollama-openai preset points at http://localhost:11434/v1. Hit it directly first, so that a later failure is unambiguously Crewship’s and not the daemon’s:
The id and the exact wording change on every run; the shape does not. The usage block is the part that matters: no usage means no tokens, and no tokens means a $0 ledger row.
5

Drive it through the codec

Now the same call, through the openai-compat codec instead of through curl. provider check builds the real provider, sends one completion, and prints what paymaster would bill for it. It is a local command: no server, no token, no workspace, and no key, because --base-url names a backend that wants none.
Read it line by line, because each one proves a different piece of the wiring:Latency is the model’s, not Crewship’s — 10–15s for a cold 0.5B on CPU is normal, and the default deadline is 60s (--timeout to change it). The reply’s exact wording changes every run.
A context deadline exceeded on the first run is usually the model loading, not a wiring fault. A cold 0.5B on a contended CPU box has been measured taking well over the 60s default while a plain curl to the same endpoint took 138s in the same window. Warm the model (ollama run <model> "") or pass --timeout 5m, and re-run before concluding anything. The exit code tells the two apart: 8 is “nothing answered at that address”, 7 is “the provider answered and said no”.
--provider takes either a registry id (anthropic, openai, ollama) or an OpenAI-compatible preset key (openai, deepseek, ollama-openai, vllm). --api-key exists but prefer the key’s environment variable where you can: an argument is visible to every process on the host through ps.
6

Confirm what it would have cost

The same rate lookup the ledger uses is available offline, so you can check a model’s price — and which of the four steps answered — before routing any spend through it:
Note the token counts are the ones provider check just reported — --in is fresh input, exclusive of --cached, which is the same convention the ledger uses.rate source is the point of the command: table is a number a human checked against an invoice, catalog is a snapshot that can be a release behind, fallback is the provider’s ceiling and therefore a deliberate over-estimate, and free is a <provider>/* wildcard row. Reading a ceiling as a real price is how a budget line stops being believed.On a pair the hand-written table also carries, the output adds a catalog says … line showing what the snapshot would have charged — which is how you see the two sources disagreeing before an invoice does.
7

Point the server at it

The daemon is also reachable as a first-class registry provider, on the native codec, with no key and no code:
aux test runs one real evaluation against that slot’s resolved model and reports the same stages the judge check uses, so a local and a hosted evaluator are held to the same bar. aux reset curator puts it back.
The curl and provider check steps exercise the openai-compat codec against Ollama’s /v1 shim. The last step exercises the ollama-native codec against the same daemon. They are different code paths to the same model, and only the second one is wired into evaluator slots today.
Two failure shapes worth recognising, because they are the ones you will actually hit:
The first is the registry and the preset list talking: the vocabulary in that message is generated from both, never written out, so it cannot drift from what the command accepts. The second is the upstream’s own refusal, preserved verbatim rather than flattened into “provider error” — and the 327ms is the tell that it never got as far as the model.

​
Pointing at a hosted provider with a real key

Three shapes, in decreasing order of how much of it is configuration rather than code.
A registry row, so it is configuration end to end. Set the key the registry names and pick the slot:
An empty key is a hard error, not a warning: the constructor would otherwise build a provider that 401s on every request, which is strictly worse than the caller falling back to a local judge with a reason in the log. To spend a vault credential instead of the server’s own key, pass --credential <name> — see Keeper.

​
The model catalog

internal/modelcatalog embeds a trimmed models.dev snapshot — the provider / model / capability / price index — so Crewship can answer “what does this model cost” without a network call.
  • What is in it: 8 of the 192 upstream providers — amazon-bedrock, anthropic, deepseek, google, mistral, openai, openrouter, xai — with every model those providers publish kept verbatim, including fields this package does not decode. Fetched 2026-08-19.
  • Why trimmed: the full index is ~4 MB and 92% of it is gateways we have no codec for. The kept set covers every provider in priceTable or the curated model lists, plus the two gateways a future provider would reach through the codecs that already exist.
  • Why offline: the catalog backs the fallback path taken when a provider cannot be reached live. A fallback that needs a network call to answer inverts its own purpose.
A corrupt snapshot degrades to “no catalog data”, never a startup failure. Every caller falls through to its own fallback — the curated model lists, the provider price ceiling — instead of taking the server down over a data file. The snapshot is decoded once, lazily, on first use: decoding ~650 KB of JSON at init would tax every binary that links the package transitively, including the CLI, where most invocations never touch it.
Refreshing it is a pure re-fetch with no editing:
jq -S in that directive sorts keys so a refresh produces a reviewable diff rather than a reshuffle. Adding a provider to the trim means editing both the go:generate line and the by-hand curl in the same comment — they are duplicated on purpose, because the comment is what a reader without go generate follows. Run the package tests afterwards: embed_test.go pins real ids and rates and is what catches an upstream repricing or a renamed id.
Three model vocabularies, one page each. The curated lists behind GET /api/v1/models answer “which ids may I type” for four providers (Model discovery). A provider’s live /v1/models answers “which ids can this key use”. The catalog answers “what does this id cost”, for eight providers. The catalog prices; it does not discover, and it is not a claim that Crewship can call the model.

​
Pricing: how a provider gets a rate

lookupPrice walks four rate sources, in this order, and stops at the first hit. Falling off the end is a fifth outcome and a different kind of answer:
1

Exact priceTable row

"<provider>/<model>", hand-verified, in internal/paymaster/pricing.go.
2

Provider wildcard

"<provider>/*" — how ollama/* and local/* say “free” with a human’s signature on it.
3

The embedded catalog

The models.dev snapshot, flattened into the same key space.
4

providerFallback ceiling

The most-expensive known tier for that provider. Every provider the registry or the catalog knows has a row here, including the two gateways (openrouter at $150/Mtok input, amazon-bedrock at $16.50), computed from the snapshot’s own maxima with tier rates included.
5

…and then $0

Reached only when the provider id itself is unknown — a typo, or a Provider.Name() nobody priced. It is not a rate; it is the absence of one, and it is the case worth chasing down.
The catalog sits third, below the hand-written table, because priceTable carries corrections a bulk import must not overwrite — the Anthropic Opus 4.7 repricing is the worked example — and because the wildcard has to keep ollama/* and local/* free. It sits above the ceiling because a ceiling that is wrong by 10× on a cheap model is the kind of wrong that makes an operator stop trusting the budget line.
A model the catalog carries no cost for is skipped, never written as a zero row. The snapshot holds 23 all-zero cost blocks — hosted models priced 0/0 upstream, which is a gap in the data and not a claim that they are free. Those fall through to the ceiling and are over-estimated, which is the safe direction. It also over-bills genuinely free OpenRouter :free tiers, and that trade is deliberate: free is a claim only ollama/* and local/* get to make, in priceTable, where a human signed off on it.

​
Long-context tiers

76 models in the snapshot publish a second rate card that takes over above a context threshold — usually 200k or 272k tokens, and up to 6.7× the base rate. Only two of them are shadowed by priceTable, so the rest reach the ledger through the catalog. Estimate receives token counts but no context axis, so it cannot know which card applies; until that signature changes, catalog_pricing.go bills tiered models at CeilingRates() — the most expensive card the model publishes. A short call on a tiered model is therefore over-estimated. That is the same call providerFallback already makes (“the most-expensive known tier, not the median”) and the same call the hand-written table made independently for the one tiered model it carries. Under-billing weakens the budget signal exactly when an operator needs it; over-billing is visible and annoying, so we take visible and annoying. The row picked is always one published tier, never a per-channel maximum assembled from several — a synthetic worst-of-every-column row would describe a price no provider charges. A caller that does know the prompt size can ask Model.RatesAt(contextToks) directly. Note that the threshold is exclusive: models.dev names these tiers “over 200k”, so a call at exactly 200,000 tokens is base-priced. cost.context_over_200k is deliberately not decoded. It duplicates one tier entry with the tier key removed, and its name lies — for 34 of the 65 models that carry it the real threshold is 256,000 or 272,000 tokens.

​
Input tokens are fresh tokens, on every codec

Response.InputToks means fresh input: exclusive of CachedInputToks and CacheCreationToks, on every provider. Anthropic reports it that way on the wire; OpenAI does not — its prompt_tokens includes cached tokens — so the OpenAI codec subtracts, clamped at zero. Without that subtraction an 80%-cached 1500-token prompt bills 2.67× the invoice, and the sidecar’s proxy path (which has always subtracted) and the in-process codec would disagree while writing to the same cost_ledger. All four OpenAI-compatible presets inherit the subtraction, including compat backends that report a cached_tokens larger than prompt_tokens — the clamp is what keeps a negative token count out of the ledger.
cost_ledger.input_tokens therefore changed meaning for OpenAI-family rows. Historic rows are not migrated, so a rollup spanning the change is not comparable across it.
Full rate card and the ledger schema: Paymaster.

​
Adding a provider

1

Add the registry row

One RegisterProvider call in registry.go’s init, at the position you want it to appear in the operator’s picker. provider_build_registry_test.go checks the row is buildable.
2

Give it a rate

Either a priceTable row (hand-verified) or confirm the embedded catalog covers the ids you will use — CatalogID is how a provider whose registry id differs from its models.dev id finds them. A row with neither bills at $0. If you added an OpenAI preset, TestPresetsResolveToAPricedProvider fails until the rate exists.
3

Wire the surfaces, if any

A new API route needs a line in internal/api/testdata/route-roles.txt, and every API endpoint needs a matching CLI command (cli_route_contract_test.go enforces it). A provider row on its own adds neither.
4

Document it

This page, plus Paymaster if the rate source is new and Environment if you introduced an env var.

​
What does not ship yet

  • No Bedrock. The AnthropicConfig seam is declared and unused; there is no SigV4 code and no AWS dependency.
  • No Google/Gemini provider. Discoverable and priced, not callable.
  • No API route for the registry. crewship provider list and crewship model price read the compiled-in tables directly; there is no providers endpoint behind them and none is needed. The vocabulary reaches the server through crewship keeper aux set --provider, the admin Keeper API and the console picker.
  • Presets are code-level. deepseek, ollama-openai and vllm cannot be selected from configuration — an evaluator slot takes only the three registry ids. crewship provider check --provider does accept a preset key, because a one-shot check is not a stored setting, but provider list will not show them: a preset is not a registry row.
  • One cache-token vocabulary. The OpenAI codec reads usage.prompt_tokens_details.cached_tokens and nothing else, so a compatible backend that names its cache hits differently records zero cached tokens and pays the full input rate — see the DeepSeek tab above.
  • Model discovery — which model ids exist.
  • Paymaster — what they cost, and the ledger.
  • LLM middleware — the call stack they plug into.
  • Keeper — the judge and the evaluator slots that consume the registry.
  • Environment — the key and endpoint variables.
  • Providers — container/storage providers, a different thing entirely.