Credentials
Crewship encrypts secret credential material at rest with AES-256-GCM. Runtime delivery depends on the credential type and CLI adapter: supported provider paths use secretless sidecar injection, while other credentials are delivered through the agent environment or/secrets files. The matrix below makes that
boundary explicit.
Credential Types
Canonical list ininternal/api (CredentialType constants). The manifest validator accepts any of these strings on a credentials[].type: field; UI and CLI use the same set.
Provider logins
A provider login is the seat an agent pays for its model with — a Claude Max setup-token, a ChatGPT plan, a metered OpenAI key — as opposed to a secret an agent uses (a GitHub PAT, a DB password). It has an owner, a plan, an expiry and, where the provider has one, a refresh flow that Crewship runs for you. The design is indocs/prd/provider-logins.md.
Creating one. POST /api/v1/credentials with type: PROVIDER_LOGIN, a
provider from ANTHROPIC, OPENAI, GOOGLE, CURSOR, FACTORY,
XAI, GROQ, OPENROUTER, DEEPSEEK, MOONSHOT, ZAI, MINIMAX,
OPENCODE, OPENCODE_GO, ZAI_CODING_PLAN, an
optional mode (subscription | api_key, inferred from the value when
omitted) and value = whatever you have: the output of claude setup-token,
the whole ~/.codex/auth.json that codex login wrote, or an API key. The
server splits it into the value (access token / setup-token / key) and the
parts refresh_token (secret, sealed), id_token (secret), account_id,
plan, expires_at, mode. A value the CLI would refuse — a bare token
where Codex wants the file, an auth.json without id_token — is refused at
submit time with the reason.
CLAUDE_CODE_OAUTH_TOKEN for
an Anthropic seat, a rendered $CODEX_HOME/auth.json with a placeholder
refresh token for a ChatGPT seat, the sidecar CredStore (or the adapter’s env
var where there is no endpoint override) for an api_key login. No part is
ever exported — <SLOT>_REFRESH_TOKEN does not exist in any container, on
any delivery path — and a login is never a /secrets file.
The login object. Every credential row that is a login — the type
itself, and the older AI_CLI_TOKEN / API_KEY rows of a model provider —
carries login with mode, provider, plan / plan_label, the owner,
expires_at, refresh (supported, status, last_at, next_at,
error), delivery (env or file and its target) and pays_for (how
many agents and crews it reaches through grants and bindings). GET /api/v1/credentials?kind=provider_login lists only those rows; GET /api/v1/agents/{id} carries pays_with, the login paying for that agent’s
adapter.
Refresh. Crewship uses the ChatGPT token’s reported expiry. Refresh tokens
can rotate on use, which is why refreshing is centralised: the server
renews the token from the sealed refresh token 24 hours before expiry, and
before any run start with less than 48 hours left, with one refresh in flight
per login at a time (a second request answers 409), a five-minute backoff
after a failure, and after three consecutive failures — or one the provider
calls permanent — refresh.status = needs_relogin plus an inbox message to
current workspace administrators. Notification visibility follows the current
role, even if the person who originally connected the account is later demoted.
A successful refresh re-renders the file into the running
containers of every Codex agent the login pays for. POST /api/v1/credentials/{id}/refresh / crewship credential refresh forces one
now. A Claude setup-token and an API key have no refresh flow
(refresh.supported = false); the expiry badge is all there is.
Reveal. A login that carries a refresh token is SEALED: nothing on it
can be revealed, by any role. The secret parts are never returned by any read
path regardless of classification.
Custom Fields (multi-part credentials)
A credential’s own value is a single secret. Plenty of credentials are not one secret: AWS static credentials are an access key id, a secret access key and a region; a service-account credential is a blob plus a filename; a login may carry a TOTP seed, a passphrase, a host or an account id. Rather than a column per shape, a credential can carry custom fields — any number of named parts, each of which is either secret or plain:
Plain fields are cleartext on purpose, for the same reason
credentials.username is: an identifier is not a secret, and storing it in the
clear lets the UI search and sort without decrypting every row while keeping
the AES-GCM surface small.
There is no way to read a secret field’s value back — not through the API, not
through the CLI. crewship credential reveal is the single disclosure path and
it applies the full reveal ceremony; rotating is usually what you actually
wanted (see Rotation Permissions).
set is an upsert: it creates the field, and replaces the value if the key is
already there. Omitting --plain stores the field encrypted — the default is
the safe one, because a field whose secrecy you forgot to state is exactly the
one you cannot afford to publish.
Rules
- Key shape —
lower_snake_case, starting with a letter, at most 64 characters (^[a-z][a-z0-9_]{0,63}$). Lowercase-only because a key becomes an environment-variable name and a file name at delivery time, whereRegionandregionwould collide once either is upcased. - Reserved keys —
value,passwordandusernameare refused. Those live on the credential itself, set through the credential’s own value rather than a field; a second copy under a field would drift away from the first with nothing to reconcile them. - Limits — at most 32 fields per credential, each value at most 64 KiB (the same cap as the credential’s own value).
- Duplicates — one key per credential.
POSTanswers409; the CLI’ssetturns that into an update. - Visibility and role — reading fields uses the same crew-scoped visibility
rule as reading the credential, and writing them requires the same role as
PATCH /api/v1/credentials/{id}(MANAGER and above). A field is never a way around the gate on the credential it belongs to.
How fields reach the agent
Fields are delivered with the credential they belong to, under a name derived from the credential’s own environment variable:SLOT is whatever the credential resolved to on this agent — the env var of
an explicit assignment, the slot of a binding (see “Slots and Bindings” below),
or (when neither exists) the credential’s name. It is never derived from the
field. That is what keeps ten accounts of one provider apart: bind GH_TOKEN
to one GitHub account and GH_ACME to another, and their account_id fields
arrive as GH_TOKEN_ACCOUNT_ID and GH_ACME_ACCOUNT_ID, each next to the token
it describes.
For the AWS credential above, assigned to the agent as AWS:
HOME, PATH, HTTP_PROXY, CREWSHIP_*,
CLAUDE_CODE_OAUTH_TOKEN, the provider base URLs) — the field is dropped,
the variable keeps the value that was already there, and the server logs a
warning naming the credential, the field key, the derived name and the reason.
A field hangs off the slot its credential is delivered under, not the raw
name — a credential called github-acme arrives as GITHUB_ACME, so its
region field arrives as GITHUB_ACME_REGION. A credential whose name yields
no variable at all is not delivered, and neither are its fields: there is no
prefix left to name them with.
Dropping is the deliberate choice: a missing AWS_REGION is one env away from
being diagnosed, while an AWS_REGION holding some other credential’s value is
not diagnosable at all — and the agent will act on it.
Beyond the naming, fields follow their credential exactly:
- Which channel — a file-delivered credential
(see below) gets one
0400file per field under/secrets/<agent>/, with.envmapping the name to the path, the same wayUSERPASSalready produces<envvar>_USERNAMEand<envvar>_PASSWORD. - Secret fields are decrypted at delivery through the same path as the credential’s own value, and they reach the agent’s environment only where that value does. Under Keeper a gated credential’s secret fields are withheld with it.
- Plain fields are cleartext from storage to container. They are identifiers, not credential material, so they are delivered even when the credential’s own value is isolated behind the sidecar proxy — a proxy can inject a key into a request, it cannot tell your CLI which region to use.
- Revocation — a soft-deleted, revoked or expired-lease credential delivers no fields either. There is no path by which a part outlives the whole.
Supported Providers
Credentials are associated with a provider for automatic injection:How Credential Injection Works
On supported reverse-proxy routes, agents never see the raw provider key. The sidecar intercepts the request and injects the credential based on the destination host:- Claude Code —
ANTHROPIC_BASE_URL=http://127.0.0.1:9119→api.anthropic.com. - Codex —
OPENAI_BASE_URL=http://127.0.0.1:9119/openai/v1→api.openai.com(the/openaiprefix keeps it distinct from Anthropic’s/v1/on the shared port, and is stripped before forwarding). - Gemini —
GOOGLE_GEMINI_BASE_URL=http://127.0.0.1:9119/gemini→generativelanguage.googleapis.com(the/geminirouting prefix is stripped before forwarding; the real key is injected as thex-goog-api-keyheader, overwriting the dummy the CLI sends).
Adapter delivery matrix
“No” in this table means the upstream secret stays in the sidecar heap. It does
not mean the request has network-layer containment: a process can still ignore
HTTP_PROXY or open a raw socket. “Yes” means any code running as the agent can
read the value, and because sibling agents share UID 1001, another concurrently
running sibling can read it too. AgentEnvCredentialExposures logs direct-env
cases so the gap is observable rather than silent.
Slots and Bindings — more than one account per provider
A credential’s name is the account (github-acme). The slot is the environment variable the agent reads (GH_TOKEN). They used to be the same string, which meant a workspace could hold exactly one GitHub account: a second one would also have had to be called GH_TOKEN, and credential names are unique per workspace.
A binding connects the two:
workspace, crew or agent.
Resolution order
agent > crew > workspace. The most specific scope that binds a slot wins, and only that one is delivered — an agent never receives two candidates for the same env var.source names the rule that won: agent_grant (an explicit crewship credential assign), agent_binding, crew_binding, workspace_binding, or crew_link — no binding at all, delivered under the credential’s own name.
Credentials with no binding are unchanged. A crew-linked credential named
CREW_TOKEN still arrives as CREW_TOKEN, and an explicit per-agent assignment still uses the env_var_name you chose. Bindings are additive: nothing is renamed, and no existing workspace has to adopt them.What counts as a legal variable name
An environment variable name is uppercase letters, digits and underscores, and does not start with a digit —GH_TOKEN, _PRIVATE, AWS_KEY_2. That is the rule the container enforces, so it is the rule everything here agrees on.
Slots and env_var_name are variables, so they are held to it at write time, and stored in the form the agent will see. You may type a dash or lowercase; the server records the uppercase form and the response shows it:
gh token, 2fa, anything non-ASCII) is a 400 on the request that chose it.
A credential’s name is not a variable — it is the account (github-acme), and it stays whatever you called it. It only becomes a variable when there is no binding and no assignment, in which case the credential is delivered under its own name. When that name is not already a legal variable, it is normalised on the way in — github-token arrives as GITHUB_TOKEN — and crewship credential resolve tells you so:
- The name folds onto one that is taken.
github-tokenandGITHUB_TOKENare two different credentials (names are unique per workspace) that want one variable. The one that asked forGITHUB_TOKENby name keeps it; the other is not delivered. Handing an agent a variable that plausibly holds a different account’s token is the failure this avoids — a missingGH_TOKENis diagnosable in one command, the wrongGH_TOKENis not. - No variable can be derived at all (
my token!). Nothing is invented for it.
One badly-named credential never costs the others, or the run. Before this, a credential named
github-token failed the variable check while its files were being written, and that failure abandoned the whole credential batch — an agent holding an SSH key and a CLI token alongside it did not start at all, and the error blamed the Docker daemon’s version. A credential that cannot be named is now skipped on its own, with a warning naming it.One slot, one credential, per scope
Within a single scope a slot points at exactly one credential. Binding a slot that is already taken in that scope returns 409 Conflict rather than replacing the existing row — silently repointing every agent in a crew at a different account is not something a write should do implicitly. Unbind first, then bind. Binding the same slot in different scopes is not a conflict; that is the whole point.The honest limit
A container has one default identity per tool.gh reads GH_TOKEN and nothing else. Ten crews with ten accounts is fine. One crew that needs two GitHub accounts at once hits the limit: the second account cannot be the default and must get an explicit slot.
gh --hostname, per-remote git config, AWS_PROFILE).
Deleting the scope deletes the binding
Bindings cascade with their owner: deleting a crew removes its crew-scoped bindings, deleting an agent removes its agent-scoped ones, and deleting a credential removes every binding pointing at it. A binding cannot outlive the thing it was scoped to and keep claiming a slot.Priority-Based Selection
TheCredStore (internal/sidecar/credstore.go) selects credentials in three steps:
- The calling agent’s grants: only credentials granted to the agent making the request are eligible. A credential linked to the crew, or bound at crew or workspace scope, is granted to every member; one granted to named agents is eligible only for those.
- Priority tier: among those, credentials with the lowest numeric priority value are selected first (lower = higher priority)
- Round-robin within tier: multiple eligible credentials at the same priority level rotate to distribute load
The CredStore is an in-memory store — credentials are never written to disk inside the container. They are loaded at container startup via stdin JSON from the orchestrator.
One sidecar serves the whole crew container, so the store is shared. Step 1 is what keeps it from behaving as if the credentials were too: the calling agent is resolved from its own per-agent route token, which the orchestrator derives per agent and which cannot be forged — it is an HMAC under a key the container never sees. On the LLM proxy’s provider path the token is derived per agent and per run: the dummy provider key carries a run-bound token, the proxy resolves the requester from it for cost attribution, and a request whose run has already ended is refused before any credential is injected — including after a sidecar restart, because revocation is replayed from the journal-backed run registry. It can be read, though: the crew shares one container and one uid, and each member’s token sits in its own environment and MCP config. This is least privilege within a shared trust domain, not isolation from a hostile peer — a crew is one trust domain by design. An agent that holds no credential for the provider gets
503 no credential available rather than a teammate’s — which matters most for OPENAI_COMPAT, whose upstream comes from the credential, so a crossed credential would have chosen the destination as well as the key (#2052).Credential Delivery Flow
1
Encrypted storage
Credentials are stored in the database encrypted with AES-256-GCM. The format is
v1:base64(IV||AuthTag||Ciphertext) where IV is 16 bytes and AuthTag is 16 bytes.2
Resolution and decryption at runtime
At run setup, Crewship resolves the agent’s explicit grants, scoped bindings and legacy crew links, filters inactive or expired grants, and decrypts the resulting secret material using
ENCRYPTION_KEY.3
Proxy-delivered credentials
Credentials selected for a supported proxy route are sent to the shared crew sidecar as JSON via stdin, not through the agent environment. The sidecar keeps them in its in-memory
CredStore.4
Directly delivered credentials
Credentials that the selected CLI must present itself are delivered through environment variables or
0400 files under /secrets/{agent-slug}/. They are readable by the agent process and are outside the secretless proxy guarantee.5
Use-time injection where supported
For a proxy-routed request, the sidecar matches the destination to a provider and injects the real authentication header. For a direct path, the CLI uses the value it already holds.
Cooldown Management
When a credential receives a429 Too Many Requests response, the CooldownManager (internal/orchestrator/) temporarily removes it from the rotation. This prevents thundering-herd problems when rate limits are hit.
Credential Security Levels
Credentials can be tagged with security levels for Keeper gating:Where the tier shows up in the console
The tier is the property that decides what happens when an agent asks for the secret, so it is on every credential surface rather than only inside the edit form:- An
L1–L4chip beside the name — in the left rail, and on both of the overview’s lists (Needs attention and Recently used). A credential whose server did not report a tier showsL?, neverL1: “we were not told” and “unguarded” are different claims. The credential’s own page carries the same fact at full width instead, as a toned pill in its header — readingTier not reportedwhere the chip would readL?. - A
Tiersection in the left rail — single-select, one row per tier, with the workspace’s count on each. Unlike every other facet on that rail, empty tiers stay on the list:L4 · critical — 0is the answer to “does anything here stop for a human?”, not a dead control. - A
Security tiersdonut on the overview, whose arcs sum to the vault. Clicking an arc selects that tier in the rail. - A
Guarded · L3+tile, counting the credentials Keeper mediates per read rather than handing to the agent for the whole run.
keeper.SecurityLevel.Tier() — unknown blast radius is
treated as the strictest tier, not the loosest.
One credential, one page
Opening a credential replaces the dashboard with that credential in full, laid out exactly like an issue: an identity card, a band of figures, then the substance in a wide column beside its properties in a narrow one. It stacks to a single column on a phone, where the rail collapses to a stub and opens as an overlay. It used to be five tabs — Overview, Fields, Used by, Audit, Settings. Tabs are a way of admitting a screen holds more than fits, and this one does not: the value, its parts, who can read it, how hard Keeper guards it and what has happened to it are the five things you weigh together when deciding whether to rotate something. Everything is on the page at once, with the audit log last.
One secret, one way to replace it. The Value card offers rotate — which
mints a new value and keeps the old one working through the grace window — and
nothing else. A plain swap with no overlap is a property of the credential like
any other, so it lives in Edit: leave the Value field empty there to keep
the existing secret. Edit is also where the credential’s icon is chosen; the
brand picker beside the name is what the rail, the dashboard and this page all
draw.
The grace window is consumed on a 401. “Keeps the old one working” was, for
a while, a promise the data layer made and the sidecar did not keep: the
previous value sat encrypted on the rotation row and nothing read it. It is now
true end to end. A crew sidecar that boots inside the window receives the
previous value beside the new one — through the same boot payload, never the
agent’s environment — and when the provider answers
401 to the new value, the
sidecar replays the request exactly once with the previous one. That is the
whole mechanism: one replay, only on a 401, only with the rotated
credential’s own previous value; a 403 or a 5xx is not retried, and a
second 401 reaches the agent. A run that succeeded this way is visible as a
network.egress journal entry naming the credential and the rotation. Ending
the window early scrubs the value on the server and a running sidecar forgets
its copy after the next successful reconciliation (polled every minute), matching
the exact rotation ID even when windows overlap. A failed metadata request
delays cancellation propagation; the original expiry still bounds fallback
locally without a server request.
The audit says who. Every row carries the actor: an agent with the avatar it
has everywhere else in the product, a person with their name, or system for a
row nobody signed. The actor used to be recorded in two different shapes — the
agent_id column on the sidecar path, and one of revealed_by / rotated_by /
created_by / approved_by / rejected_by in metadata on the human ones — so
the timeline could say what happened and not who did it. GET /credentials/{id}/audit now resolves all of them into actor_kind, actor_id
and actor_name. Ten events are shown at a time with the total beside them
(10 of 37); a credential read every few minutes is otherwise fifty rows of
“USE · 3m ago”. crewship credential audit <name> prints the same two columns
(ACTOR, WHO).
A sidecar fetch is the one event with no agent behind it — a sidecar serves a
whole container, not one agent — so it is attributed to the crew that owns
it. A sidecar polling workspace-wide has no crew either, and those rows say
system, which is the honest answer rather than a guess.
Reveal says why it is unavailable. It has four gates — the workspace switch,
the MANAGER floor, the credentials:reveal capability, and the SEALED
classification — and failing any of them used to render nothing at all. Each
gate is a different fix, so the card names the one that is shut.
Audit and rotation history are still gated on role, not on layout: GET /audit
is MANAGER+ because it exposes the IPs behind admin actions, and rendering an
empty timeline for someone who is merely not allowed to read it would be a
false statement rather than a missing section.
The Credentials overview
Opening Credentials with nothing selected shows the vault’s dashboard, not a table: what is active, what is guarded, what is missing a CLI, what expires inside 30 days; then the tier donut, the queue of what needs attention, the breakdown by type, and either what expires next or what was read most recently. Picking a credential in the rail replaces it with that credential’s detail. There is no credential table. The left rail is the list — searchable, filtered, sorted, and multi-selectable — so the main pane never shows the same names a second time. What the old table’s columns carried moved to where each fact belongs:
The cards report the whole workspace on purpose; the rail’s filters narrow the
rail, not the dashboard. A donut that redrew itself to one slice the moment you
clicked a slice would answer nothing.
Narrowing the list
The rail’s search box matches a credential’s name, account label, description, tags and the names of the agents holding it. Everything else is behind theFilter button, and each group draws its rows with the thing they are about
rather than one repeated glyph:
- Brand — the provider’s own mark. What the create wizard sets, and what the icon on every row already shows.
- Shape — token, login, certificate. The wizard’s first question, and the one to reach for before an expiry sweep.
- Scope — a workspace icon, then each crew’s own tile and colour.
- Assigned to — the agents holding at least one credential, each with the same avatar it has everywhere else in the product. Selecting one answers “what can this agent read?”.
- Tag — every tag in use, with the number of credentials carrying it.
Filter button is how many values are
narrowing the list.
Status and tier stay in the rail itself rather than in the dropdown: both are
single-select, both are bounded, and both are asked on arrival.
Selecting several credentials is a mode, not the rail’s resting state:
press Select in the Credentials header and the rows grow checkboxes;
press Done (or Cancel on the floating bar) and the selection goes with it.
A checkbox on every row all the time says the list is a thing you tick, when it
is overwhelmingly a thing you click — and it leaves a bulk delete one mis-click
from every secret in the vault.
Provider logins
A provider login is not a secret an agent uses — it is the seat an agent pays with: a Claude Max setup-token, a ChatGPT plan, a metered OpenAI or Gemini key. It has an owner, a plan, an expiry and, where the provider exposes them, two quota windows, and Crewship keeps it valid. None of that fits a row that says “ai cli · L1”, so provider logins have their own tab. Provider logins are administrative: only a workspace OWNER or ADMIN sees, creates, edits, refreshes or revokes one — including the olderAI_CLI_TOKEN
/ API_KEY rows of a model provider. Below that role the Providers tab is
empty, crewship credential list --kind provider_login returns nothing, and
any per-id request answers “not found”; a credential.create grant does not
open this door. Delivery to agents is unaffected — the seat still pays for
the agents it is bound to.
The Providers tab
Credentials → Providers lists one row per login: the brand mark, the name you gave it (“Claude Max · pavel”), the provider, mode and where it lands (CLAUDE_CODE_OAUTH_TOKEN, or file .codex/auth.json), then the plan, when
it expires (with ”· auto” when Crewship refreshes it itself), what it pays for
(“3 agents · 1 crew”, or “nobody yet”), the 5-hour and weekly windows as bars,
and a status pill. Four tiles above the list count the whole tab — Seats,
At limit, Expiring ≤ 30 d, Unassigned — and clicking one narrows the
list. The rail on the left carries the facets a seat is asked about: Status,
Provider, Mode (subscription / API key) and Owner.
Provider logins are not counted in the Overview’s “N secrets”; the page
header says “12 secrets · 5 provider logins” and, when it applies, “1 at
limit”. Every older AI_CLI_TOKEN or API_KEY row with an AI provider shows
on the tab too — the server derives its login facts — without migration.
A fact the server does not have is drawn as unknown, never as a number: a
provider whose windows cannot be read says “quota not readable for this
provider”, an unknown expiry prints ”—”.
What each status means
A seat’s page
Opening a row shows, above the usual value, fields and audit sections:- Seat — owner, plan (and that it was read from the token claim), the provider account id, and how it bills (flat-rate seat, or metered per token through the sidecar).
- Quota — both windows and when they reset, or the sentence saying the provider does not expose them.
- Validity & refresh — when the access token expires, the last and next
refresh, and the refresh token shown as SEALED: it never leaves the
server, is never delivered to a container and cannot be revealed. Refresh
now runs one refresh immediately (
crewship credential refresh <id>); a second one while it runs is refused, one at a time by design. A seat with no refresh flow says so — “paste a new token before<date>”. - Delivered to the agent as — the environment variable, or the file in the
agent’s HOME (
.codex/auth.json, mode 0600) rendered with the real access token and a placeholder refresh token, rewritten at every run start and after every refresh, removed on revoke. A subscription seat’s traffic goes through the sidecar tunnel, so model policy is not enforced in subscription mode. - Pays for — the agents and crews on this seat, with Assign to agent,
which writes an AGENT-scope binding (the same row
crewship credential binding create --scope AGENTwrites).
403 as an invalid login — verifying a subscription
needs a controlled agent run), Re-login and
Revoke.
Adding one, and signing in with a code
Add a credential → Provider login asks for the provider, then how the seat pays (Subscription or API key), then the login itself, then the owner — the person whose seat it is, defaulting to you; seats are per person under both providers’ terms. Anthropic takes the output ofclaude setup-token; a
key is pasted as a key. For an OpenAI subscription there are two ways in:
- Import from Codex CLI — paste
~/.codex/auth.jsonfrom a machine wherecodex loginsucceeded. The server splits it into parts and seals the refresh token on save. - Sign in with a code — Crewship asks OpenAI for a one-time code and shows
it, large and copyable, with the page to enter it on. Open that page on any
device, type the code, approve. Crewship polls the provider at the interval
it names; when the sign-in lands the login exists — refresh token sealed —
and the wizard moves on to who pays with it. An expired or denied code offers
a new one. The same flow from a terminal is
crewship credential login --provider OPENAI, which waits and prints the result.
Pays with, on the agent
An agent’s Config tab carries Pays with under the CLI adapter: the seats of the adapter’s provider with plan, owner and status; seats of another provider visible but greyed as “wrong provider” (Codex CLI cannot pay with a Claude login, and the mismatch is said here rather than as a 401 inside a run). Choosing one writes the agent’s own binding, which wins over the crew’s and the workspace’s. If the chosen seat is at its limit and no other seat of that provider is in scope, the row warns that the agent’s next run waits.Account groups (provider pools)
Account groups on the Providers tab put several seats of one provider and one billing mode into a named, ordered set — the shape in which, once bound, a seat at its limit hands its turn to the next. The API and the CLI call the same object a provider pool:crewship credential pool create|list|get|update|delete
(CLI reference,
API reference).
A group is a definition only for now — it does not bind seats to an agent,
grant them or drive failover yet — and it is OWNER/ADMIN-only like the seats
in it. Accounts owned by different people can be grouped only with explicit
consent, because seats are per person under the providers’ terms.
Listing & Filtering Credentials
crewship credential list returns the workspace’s credentials. For large
workspaces it supports server-side filtering and cursor pagination:
--search and --tag are applied on the server. When more results remain and
you didn’t pass --all, the CLI prints the cursor for the next page to stderr;
pass it back with --cursor <value>.
Under the hood these map to the GET /api/v1/credentials query parameters
search, tag, limit, and cursor. The endpoint is backward compatible:
without paginate=true it returns the plain array it always has; with it, the
response is a { "credentials": [...], "next_cursor": ... } envelope
(next_cursor is null on the last page).
The web Credentials page follows the workspace selected in the top-bar
switcher, so users in multiple workspaces can manage each one’s credentials. It
has no table — the left rail is the list; see One credential, one
page.
Adding Credentials
Via the UI
Open Credentials and click Add secret. The flow asks three questions, in this order:- What shape is it — Token, Login, Key pair, SSH key, File or Certificate. The shape decides which boxes you fill; there is no brand catalog to hunt through. Any of the thousands of services out there fits one of these six, and anything else fits a custom field.
- The values the shape implies, plus a name for the account
(
github-acme, notGH_TOKEN— see Slots and Bindings). Parts marked plain in the table above are stored in the clear; everything else is encrypted and never readable again. - Who gets it, and under which variable — the whole workspace or selected crews, then the slot: the environment-variable name the container will see. Leave the slot empty and the credential is delivered under its own name, which is the pre-slot behaviour.
ghp_…, sk-ant-…, AKIA…) picks up the brand
icon and suggests a slot name. That is a hint, not a gate — an
unrecognised value is completely normal and nothing blocks on it.
Choosing the slot requires workspace-admin rights (POST /api/v1/credentials/bindings is OWNER/ADMIN). A MANAGER can still create the
credential; an admin can bind it to a slot afterwards.
Secret values (and OAuth refresh tokens) are capped at 64 KiB per value; ENDPOINT_URL values are capped at 2048 bytes. Oversized values are rejected with a 400 at submit time. A credential may carry at most 32 custom fields.
The Credentials list
The list has a left rail in the same shape as Integrations: Status (everything / needs attention / missing tool) and Tier in the rail itself, then Brand, Shape, Scope, Assigned to and Tag behind theFilter button, each with its own count.
The Readiness column is the one that is not about the vault. It answers
“does the crew’s container actually have the CLI this credential is for?”,
from GET /api/v1/crews/{crewId}/credential-readiness, and it has three
states — not two:
The agent overview asks the other readiness question — “does a credential the
runtime will actually deliver authenticate this agent’s model?” — from
GET /api/v1/agents/{agentId}/credential-readiness, and warns No model
credential only when the answer is missing. It cannot be read off the
agent’s credential list: one crew-scoped GH_TOKEN binding makes that list
non-empty for every agent in the crew, and for CLAUDE_CODE the model key in
the env is a dummy by design (the sidecar injects the real one). The same
verdict is crewship agent credential-readiness <agent>; see the
agent CLI page.
Revealing a value from the UI
The credential detail sheet offers Rotate and show the new value as the primary action and Reveal the existing value… as the secondary one. That ordering is deliberate: most reasons to want a value are really reasons to replace it, and a control that is used rarely is a control that keeps working. The Reveal action only appears when all of the following hold, which is exactly what the server checks:- reveal is enabled for the workspace (Settings → Access & Secrets),
- your membership holds the
credentials:revealcapability — theadminpreset includes it, and an OWNER or ADMIN whose membership has no explicit capability set holds it by default; an explicit set is authoritative, so it can be revoked from an admin and granted to a MANAGER, - your role is MANAGER or above, and
- the credential is not
SEALED.
Settings → Access & Secrets
Workspace-wide policy lives in Settings → Access & Secrets, not on any one credential’s page:- the reveal switch — read by MANAGER and above, changed by the OWNER only (an ADMIN is refused by the API, so the switch renders read-only for them and says why),
- who holds
credentials:reveal, with a warning once it is spread wider than a couple of people, - what each classification means and who may move a credential between them (MANAGER+ to raise, OWNER/ADMIN to lower — lowering is journaled).
Via the CLI
--crews accepts crew slugs or IDs and is the CLI equivalent of the UI’s crew-scoping picker (it writes the credential_crews junction). Supplying crews sets the credential’s scope to CREW; omit it for a workspace-wide credential. --scope WORKSPACE|CREW is available to set the scope explicitly, but it’s normally inferred from --crews.
--scope only accepts WORKSPACE or CREW (case-insensitive) and is rejected client-side otherwise; on credential update it also rejects an empty value. Note that the server always derives scope=CREW when --crews resolves to a non-empty list, even if --scope WORKSPACE was passed explicitly — the CLI prints a warning to stderr in that combination, but the server’s derived scope wins.
Sign in with a code
A subscription login for a model provider does not have to be pasted. For OpenAI (a ChatGPT Plus / Pro / Business plan, used by the Codex adapter) the server can run the provider’s device-code sign-in itself:auth.json that codex login would have written, and
stores it as a PROVIDER_LOGIN credential owned by you, named after the plan
and the account (ChatGPT Plus · jana@unify.cz). The command then prints the
credential the way credential get does. Assign it to a Codex agent like any
other login; it is delivered as a rendered file whose refresh token never
leaves the server — see Codex CLI.
Because the server does the waiting, a closed terminal does not lose the
sign-in: --no-wait prints the code and a device id and returns, and
crewship credential login-status <device-id> reports pending, complete,
expired or denied later. A code is valid for 15 minutes, and a server
restart in the middle resumes the wait. Starting a sign-in requires the
workspace OWNER or ADMIN role; credential.create alone does not grant
provider-account access. Starts are limited to five per person per ten minutes.
Enter the code only for a sign-in you started yourself: a code shown to you
by a page or another person is how an account gets bound to a workspace it
does not belong to.
Via Seed Data
For development, set environment variables before seeding:sk-ant-oat) vs API keys and creates the appropriate credential type.
Rotation Permissions
Rotating a credential (and cancelling a rotation’s grace window early) requires theOWNER/ADMIN role or the credential.rotate member capability. Granting that capability to a MANAGER or MEMBER lets an oncall user rotate a leaked token without giving them blanket vault access — see the capability list in the Workspaces API reference.
Credential Assignment
Every credential belongs to a workspace, but an agent’s delivery set is resolved from three sources: an explicit per-agent grant, the winning agent/crew/workspace binding for a slot, and a legacy crew link. Explicit grants take precedence for the same credential. Provider-based auto-assignment creates explicit grants in selected agent-creation flows; it is not the only way a credential reaches an agent. Two explicit-assignment modes exist:Short-lived leases (TTL)
An assignment is normally a standing grant — long-lived, reused across every session until it is unassigned. A stolen standing grant stays valuable indefinitely. A lease flips that: the grant is issued with a short TTL, and once it lapses it is refused at credential-injection time (fail-closed) — the agent must be re-granted access. The durable secret always stays in the vault; the grant is the only thing that expires. Add a lease by passing--ttl to credential assign:
--ttl accepts Go durations (30m, 2h, 24h), capped at 30 days — a
multi-month “lease” is a standing grant in disguise and is rejected. Omit --ttl
for a standing grant.
Enforcement is at the injection point, not just the UI, and it covers every
path that hands a grant’s plaintext to an agent:
Server-side gates decide what to deliver at the instant of delivery. A
container then holds that plaintext for its whole life, so leases also travel
with the delivered credential:
- The boot payload carries each credential’s
lease_expires_at. - The sidecar’s credential store refuses a credential whose lease has lapsed the moment it lapses (not on the next sweep), and the credential reaper evicts it from memory within one ~60s interval so the plaintext stops being resident.
- Unlike revocation — which needs a
crewshipdround-trip and therefore fails open, so a transient blip cannot nuke a working key — lease expiry needs no round-trip and therefore fails closed. An unreachable server is not an excuse to keep serving a lapsed lease.
The lease lives on the per-agent grant. A credential whose
scope is
WORKSPACE (or that is granted to a crew via credential_crews) is still
reachable through that path, which carries no TTL — a lease narrows one
agent’s grant, it does not retro-scope a workspace-wide credential. Scope the
credential to the agents that need it if you want the lease to be the only way
in.EXPIRED <timestamp>. SOURCE says which event
minted it — see the next section.
Auto-issued leases (on approval)
--ttl leases one grant by hand. Auto-lease does it for you: with a TTL
configured for the workspace, every approval re-issues the grant as a lease, so
credential access decays instead of accumulating.
SOURCE in parentheses) when:
- the Keeper ALLOWs a
/keeper/requestor/keeper/executecall (keeper_allow), or - a human approves an agent-proposed
CREDENTIALescalation (escalation_approve).
- Opt-in. Default off. Nothing changes until you set a TTL.
- L3/L4 only. L1/L2 self-service credentials are delivered to the agent for the whole run (they are how it calls its own model), so they are never auto-leased — expiring one mid-run would break the agent’s own work rather than contain an attacker.
- Never shortens a longer lease. A grant you leased with
--ttl 7dkeeps its 7 days; only a standing grant or a shorter lease is rewritten. - Bounded. Minimum 60s (a shorter lease can lapse inside Keeper’s own evaluation, denying the request that authorised it), maximum 30 days.
LEASED row to the credential’s audit timeline and a
credential.lease_issued entry to the journal, carrying the source, the
resulting expiry and the authorising request id — so “why did this credential
stop working at 14:32?” is answerable.
With auto-lease on, approving an agent-proposed credential escalation also
creates the proposing agent’s grant (leased) if it had none. With auto-lease
off, approve behaves as it always has: it activates the credential and creates no
grant.
Asking a human for a credential
An agent that needs a secret only a human has — a database password, a partner API token — asks for it. It raises aCREDENTIAL escalation whose metadata
names the credential and says what it is for, and carries no value:
-
The Keeper judges the ask first, before anything is staged. It sees the declared
name, type, tier and purpose and the agent’s recent conversation — never a value, because
an ask has none — and answers with the same judgement it applies to credential access:
- ALLOW → the ask is staged and routed to a human, as below.
- DENY → nothing is staged, no inbox card, no human interrupted; the agent’s
/escalatecall returns{"status":"DENIED","reason":"…"}and the denial is recorded in the journal. The agent is expected to read the reason and move on, not re-ask with reworded wording. - ESCALATE (and any judge outage) → staged and routed to a human as normal, with the judge’s note attached to the card so the reviewer sees why it was flagged.
-
A credential named
PG_PASSWORDis staged in the vault immediately, in statusREQUESTED— the name, type, tier and purpose the agent declared, and no value. It is marked handle-only (below). Like aPENDING_APPROVALrow it is filtered out of every delivery path, so nothing can use it yet. The escalation links it. -
The inbox card says Credential requested: PG_PASSWORD and, on the crew’s escalations
panel, offers a masked input. What the human types goes straight into the vault
—
POST /api/v1/escalations/{id}/supply, or from a terminal:The value is read from stdin only. There is no flag for it, so it never lands inargv, the process table or shell history. -
In one transaction the credential becomes
ACTIVE, is granted to the asking agent under the name it asked for (leased, when the workspace has auto-lease on), and the escalation is resolved with nothing in its resolution column. -
The agent’s
/escalatecall returns:A handle, never the value. The agent uses it through/keeper/execute— the value is injected into that one command inside the container and scrubbed from its output — exactly like a Keeper-guardedSECRET.
security_level in the ask is the agent’s proposal; the approver sees it and the
human supplying the value can override it (--security-level). hosts and
purpose are review metadata for the approver — shown on the card, stored as the
credential’s description, not enforced: there is no per-credential egress policy, and
a list that looked enforced but was not would be worse than none.
A late answer still lands. The agent waits up to 5 minutes; the human has 7 days. If
the human answers after the agent moved on, the grant is a row, not a message: the
agent finds
PG_PASSWORD in its .env on its next run and does not have to ask again.
This is the property the old flow could not have — a value delivered after the wait
ended had nowhere to go.Handle-only credentials
credentials.handle_only says the agent may use this value, never read it. Every
credential created from an ask is handle-only — the flag is set by the ask and by the
human’s supply, and only there: no create or update field and no CLI flag sets it on
an ordinary credential. It is enforced at the one place every delivery path derives its set from
(loadDeliveredCredentials) — the boot resolver, the peer-query loader and the
sub-agent loader alike — and it does not depend on Keeper being enabled: it is a
property of the secret, not of the instance. A handle-only credential is delivered as
its name with no value, listed in the agent’s [KEEPER] prompt block, and reachable
only through /keeper/execute or the sidecar proxy.
What a CREDENTIAL escalation never carries
/resolverefuses text.PATCH /api/v1/escalations/{id}/resolveon aCREDENTIALescalation answers 400 to anyresolution, approve or reject. The only text a human ever typed there was the secret; the supply endpoint is where a value goes, andapprove/rejectneed no words. Approving an ask that still has no value is 409.escalations.resolutionis never written for aCREDENTIALescalation. It used to hold the human-typed secret, encrypted — a second vault with no tier, lease, rotation or audit timeline, whose one reader handed the plaintext back to the agent. Historical rows now hold the marker[credential submitted]; the ciphertext is gone.- The agent-facing wait body has no
resolutionkey on aCREDENTIALanswer — not an empty one, none. What it has iscredential.
Agent-Proposed Credentials (pending approval)
Agents cannot activate a credential on their own — the lifecycle requires a named human, and the/secrets/{agent-slug}/ files carry no authority — they are plain
0400 copies on the crew tmpfs, and rewriting one does not register anything
in the vault. But an agent that generated a secret for the crew — say
a password for a database it just provisioned — can propose it, and a human approves
it with one click.
The agent raises a CREDENTIAL escalation through the sidecar, carrying the proposed
credential as JSON in the metadata field. Send the body over stdin (--data @-) so the
secret never lands in the shell history or process arguments:
Attribution comes from the presented per-agent bearer token, not the
from
field. Each agent receives $CREWSHIP_AGENT_TOKEN (derived as
HMAC(master, workspace‖agent)) in its environment and MCP config. The shared
sidecar matches it against the crew roster and overrides caller-supplied
identity fields, so changing from alone cannot impersonate a peer and an
unknown token is refused. This is token attribution, not an intra-crew process
identity boundary: siblings share UID 1001 and the process namespace, so one
that reads a peer’s token can replay it and inherit that attribution. A missing
token is refused whenever the crew has a token roster; only a genuinely
token-less legacy deployment falls back to membership-validated request fields.- The value is encrypted and stored immediately as a credential with status
PENDING_APPROVAL. It shows up in the Credentials page (amber Pending approval badge, under Needs attention) and as an item in the inbox, but it is never delivered to any agent while pending. - A human Approves — from the inbox (one click), the crew escalations panel, or
crewship escalation resolve <id> --action approve(no--resolution: aCREDENTIALescalation takes no text) — and the credential flips toACTIVE, attributed to the approver. Only now can the crew use it. - Or Rejects (
--action reject) and the proposed credential is discarded.
/escalate call blocks until the human approves or rejects (up to 5 minutes).
If the agent does not have the value itself, it asks instead — see
Asking a human for a credential.
A free-text ask (no structured metadata) is still accepted: the human names the
credential when supplying it (crewship escalation supply <id> --name PG_PASSWORD).
Provenance is preserved either way: the credential records the agent as proposer or
requester and the human as approver.
When a proposal can’t be staged
The proposed secret is discarded (never persisted in the clear) the moment it is read, so the escalation never reflects success it didn’t achieve:- Name already in use or unknown credential type — recoverable. No credential is
staged, but a plain escalation is still raised with a note explaining why, so a human
can supply or rotate the value manually. The call returns
201. - No workspace owner to approve the proposal, or a vault/encryption error — a hard
failure. No escalation is recorded and the call returns
503with a message the agent can act on (retry, or ask an operator to configure a workspace owner). This avoids a phantom “pending approval” that no one can act on while the secret is already gone.
Segregation of duties (second approver)
By default, any workspace MANAGER+ can resolve aCREDENTIAL escalation below L4 —
including one raised by an agent they created themselves. Regulated workspaces can opt in to
a strict four-eyes rule for every tier: the person who proposes a credential (via their
agent) can never also be the one who approves it.
Enable it per workspace (requires OWNER or ADMIN):
crewship escalation resolve (and the inbox Approve/Reject button) is
refused with a 403 for the user recorded as the owner of the agent that raised the
CREDENTIAL escalation (agents.created_by_user_id — whoever created that agent). This is
checked for every resolution action (approve, reject, redirect), not just approve, and
before any row is mutated. Every blocked attempt is written to the audit journal (a
keeper.decision entry naming the segregation_of_duties rule and the blocked user).
OWNER is not exempt. This is a strict segregation-of-duties gate, not a permission
check — even a workspace OWNER is refused if they are also the recorded owner of the agent
that raised the escalation. Someone else with MANAGER+ role must resolve it instead. The
rule only applies to
CREDENTIAL escalations; TEXT/LINK escalations are unaffected. If
the raising agent has no recorded owner (a legacy agent created before this attribution
existed), the rule fails closed: resolve and supply are refused with a 403 (“a second
approver is required, but the initiating identity is unknown”) rather than waved through,
and the inbox reports the requirement all the same. Such an ask needs an agent with a
recorded owner, or a workspace where neither the toggle nor the tier forces the rule.Scope limit — identity is agent ownership, not the driver. “Initiator” is the agent’s
recorded owner (
created_by_user_id), not necessarily the human who drove the agent to
raise the escalation. If user A owns an agent but user B drives it (via chat) to propose a
credential, the rule blocks A — user B could still self-approve. This covers the common case
(owner drives their own agent); tightening it to the actual requester requires recording the
driving user on the escalation at raise time (planned follow-up).Two guard rails. Enabling the rule on a workspace with fewer than two members who can
approve (OWNER/ADMIN/MANAGER) still succeeds but returns a warning — with only one
eligible approver a credential raised via their agent could never be resolved by someone
else. And
crewship credential rotate (POST /credentials/{id}/rotate) is refused with a
409 for a credential still PENDING_APPROVAL, so rotation can’t be used to activate an
agent-proposed credential outside the four-eyes flow — approve or reject it via its escalation.GET/PUT /api/v1/admin/keeper/governance) — it is a distinct concern (who may approve a
credential escalation, not behavior monitoring) but reuses the same OWNER/ADMIN-gated,
journal-audited settings surface. Default is off.
File-Based Secrets
For non-LLM credentials (database passwords, API tokens), Crewship writes one file per credential to/secrets/{agent-slug}/:
.env file maps environment variable names to file paths for tooling compatibility.
Lifecycle: in-memory, per-run, removed on completion
/secrets is an in-memory tmpfs mount (16 MiB, mode 0700, owned by the agent UID) — never a host directory. Credential files exist only in RAM inside the running container: they never touch the host disk, never survive a container stop, and can’t leak into host backups or archives. Requires Docker Engine 26+ for the tmpfs ownership options; crew containers created by older Crewship versions (which used a host bind mount) are automatically recreated with the tmpfs on their next use, and the legacy host-side directory is deleted.
The files are also per-run: Crewship rewrites them at the start of every agent run and removes /secrets/{agent-slug}/ again when the run finishes (refcounted, so overlapping runs of the same agent are safe). Between runs there is nothing under /secrets/{agent-slug}/ for another container process to read.
SECRET delivery depends on Keeper
SECRET credentials follow the Keeper state, and the file path mirrors the
env-var path exactly:
- Keeper disabled — the
SECRETis delivered as a0700/0400file under/secrets/{agent-slug}/(and mapped in.env), just likeCLI_TOKENandGENERIC_SECRET. This is the legacy default and is unchanged. - Keeper enabled — the
SECRETis not written to the agent’s filesystem at all, and it is not injected as an environment variable. The agent’s system prompt states it does not have the value in its environment; to use it the agent must request it through the Keeper API (/keeper/requestreturns only anALLOW/DENY/ESCALATEdecision;/keeper/executeruns one command with the secret injected into that command’s environment), which enforces access control and writes an audit trail without returning the raw value. Withholding happens once, at the config resolver — the plaintext is blanked before it ever leaves the server process, so the env, file, and MCP delivery paths gate on Keeper as defense-in-depth rather than as the sole line. The resolver logs each withheld credential atWARN(env-var name only, never the value), so the gate is observable rather than a silent skip. There is no file to read and nothing under/secrets/{agent-slug}/for a compromised process to scrape.
internal/credpolicy), not by a SECRET-shaped
special case scattered across the delivery paths — so a new credential type is
governed by one row, and an unclassified type fails safe (withheld and not
delivered). Today the gate is SECRET-only: CLI_TOKEN, GENERIC_SECRET,
USERPASS, SSH_KEY, and CERTIFICATE are always delivered as files
regardless of Keeper state — CLI tools and generic secret consumers read them
from disk, and the prompt makes no withhold claim for them.
An unclassified type’s withholding is stricter than SECRET’s: SECRET has an
explicit delivery channel (a file), so with Keeper off it still falls back
to the legacy env-var path. An unclassified type has no delivery channel at
all — it stays withheld regardless of Keeper state, and with Keeper off there
is no /keeper/request path to recover it either, so it is simply
undeliverable rather than gated. If a credential row predates the type enum
(the credentials.type column was unvalidated free text for most of the
product’s life) or names a type that was never given a credpolicy row, it
now goes silently missing from the agent’s environment; the orchestrator logs
a WARN naming the credential’s env-var name and type when this happens, so
the fix is to add the missing row rather than to chase a mysteriously absent
credential.
The sidecar-mode env builder selects a credential for injection through
several independent rules, not one — an OAuth token, a CLI tool’s token, an
adapter-recognized provider API key, and the SECRET legacy fallback are each
matched differently (by type, by value shape, or by env-var name). Every one
of those rules now consults the credpolicy row before it appends a value,
so an unclassified credential cannot reach the env by accidentally matching
one of them (e.g. an env-var name an adapter recognizes, or a value that
happens to look like an OAuth token) — it is withheld the same way regardless
of which rule would otherwise have picked it up.
The non-sidecar env builder (BuildEnvVars() — see Non-Sidecar
Mode below) consults the same
credpolicy row before it writes anything, for the same reason: this path is
not only what runs with the sidecar disabled instance-wide, it is also what a
worker sub-agent uses (SkipSidecar) when the crew’s sidecar is already
running for another agent in the same container. Without this check, an
unclassified credential’s plaintext could be withheld from the lead agent’s
sidecar-built environment and still land in a worker sub-agent’s — the same
credential, the same Keeper state, two different outcomes depending only on
which agent happened to dispatch the exec. BuildEnvVars() gates on the
credpolicy row alone (it has no keeperEnabled parameter and never has):
the fallback row’s DeliveryNone means no channel in either Keeper state,
so no Keeper plumbing is needed to close this gap. This does not change
this path’s pre-existing, unrelated behavior for classified Keeper-gated
types (SECRET) — that legacy env delivery is out of scope here; see
Non-Sidecar Mode.
MCP config references are covered too. The withholding above spans all
three credential-delivery paths — the credential files, the direct env
builder, and MCP env injection. If a
SECRET’s env-var name is referenced by an
MCP server config (e.g. Authorization: Bearer ${DB_PASSWORD}), Crewship does
not inject the value into that MCP process’s environment when Keeper is
enabled: the SECRET is withheld exactly as it is on the file and direct-env
paths, so it stays out of /proc/self/environ and out of the /keeper/* audit
bypass.A consequence follows: because the MCP process runs inside the agent container
and cannot call the Keeper API itself, an MCP server that genuinely needs the
value will not receive it while Keeper is gating that SECRET. If a server must
authenticate with a credential, give it a non-SECRET credential type
(CLI_TOKEN, GENERIC_SECRET, …), or run that server without Keeper-gating the
secret it needs. The gate stays SECRET-only and Keeper-only — non-SECRET
types referenced by MCP configs are injected as before, whether Keeper is on or
off.The trust boundary is the crew, not the agent
Every agent process in a crew container runs as the same principal — UID 1001. The0700 mode on /secrets/{agent-slug}/ protects it from the sidecar (UID 1002), not from sibling agents: to the kernel, all agents in one crew are the same user, so any agent can read and write every sibling’s /secrets/{slug}/ files, home directory (/crew/agents/{slug}/), output, and MCP configs, and can signal every sibling’s processes. The per-run write/remove lifecycle above shrinks the at-rest window, but while two agents run in parallel there is no intra-crew confidentiality.
This is the runtime’s deliberate posture, not an oversight: the enforced isolation boundary is the crew. Each crew gets its own container, its own in-memory /secrets tmpfs, its own home volume, and its own sidecar, and a sibling crew reaches none of those four (the sidecar listens on loopback only). That is resource isolation, not network isolation: every crew shares one Docker bridge, so a crew’s declared service containers are reachable from another crew. Within a crew, agents are assumed to trust each other, the same way processes owned by one Unix user trust each other.
Revoking a file-based secret
Deleting a credential (crewship credential delete <name>, or the UI) removes its /secrets/{agent-slug}/ file(s) from every running crew container that mounted it — immediately, not just on the next boot. The same removal fires when the sidecar reports a credential’s status as REVOKED (e.g. the upstream provider invalidated it). The removal runs server-side, exec’d inside the container as the agent UID (1001): that’s the only principal that can unlink inside the agent’s 0700 secrets directory, so the sidecar (UID 1002) can’t do it and the server does. A successful delete has already cut off live access.
A multi-part credential’s custom field files go with it — the removal covers every file the credential delivered, not just its primary one, so a revoke never leaves a secret access key behind while reporting the credential gone.
It’s best-effort: a stopped container is skipped (nothing to remove; the revoked secret won’t be re-written on the next boot because it’s filtered from every delivery path). The .env map keeps an inert reference to the removed file until the container next boots — harmless, since the file is gone. If you need a guaranteed clean slate, recreate the crew container.
Three Credential Injection Modes
Crewship supports three distinct modes for delivering credentials to agents, depending on the container configuration and credential type.1. Sidecar Proxy Mode (default)
1. Sidecar Proxy Mode (default)
The standard and most secure mode. Credentials are piped to the sidecar process via stdin as JSON at container startup. The sidecar holds them in an in-memory This is a workspace-wide flag, not per-crew: it affects every privileged crew in the workspace. Turn it off the same way (
CredStore and intercepts outbound HTTP requests, injecting the appropriate authentication headers based on the destination host.- Credentials on a proxy-routed path never appear in environment variables or on disk. Adapters with no routable endpoint,
AI_CLI_TOKEN,OAUTH2and file-delivered types are delivered directly and do — see the adapter matrix - The sidecar runs as UID 1002, inaccessible to the agent (UID 1001)
- Supports priority-based selection and round-robin rotation
--privileged (e.g. Docker-in-Docker) drops no-new-privileges and CapDrop:ALL for the whole container — the same UID 1001/1002 boundary that isolates the sidecar’s memory (and the credentials loaded into it) from the agent no longer holds. A compromised or prompt-injected agent in a privileged container can read the sidecar’s process memory directly.Crewship fails closed by default: the agent-config resolver omits credentials entirely for a privileged crew, and provisioning logs a WARN (#1032) so the trust downgrade is visible in ops. To keep credential injection working for a privileged crew anyway — accepting that the isolation boundary is gone — a workspace OWNER or ADMIN opts in explicitly:--allow-privileged-credentials=false) once the crew no longer needs it.2. Non-Sidecar Mode (legacy)
2. Non-Sidecar Mode (legacy)
When the sidecar is disabled, credentials are injected as environment variables using
BuildEnvVars(). This mode is less secure because credentials are visible in the process environment.This is also the path a worker sub-agent takes when dispatched with SkipSidecar — the crew’s sidecar is already running for another agent in the same container, so the sub-agent’s own exec doesn’t start one and falls back to BuildEnvVars() instead of the sidecar env builder.- API keys are set as
ANTHROPIC_API_KEY,OPENAI_API_KEY, etc. AI_CLI_TOKENcredentials withsk-ant-oatprefix are mapped toCLAUDE_CODE_OAUTH_TOKEN- An unclassified credential type (no
credpolicyrow — the fail-safe fallbackDelivery: DeliveryNone) is withheld here too, in either Keeper state, matching every selector in the sidecar env builder. This keeps the “forgetting to classify a type leaks nothing” promise regardless of which of the two env builders picks a credential up. - This is not the same as Keeper gating:
BuildEnvVars()has nokeeperEnabledparameter and does not consult it. A classified, Keeper-gated type (SECRET) is still delivered directly here exactly as it always has been — this mode predates Keeper and was never changed to honor it. ItsSECRETplaintext is visible in the process environment here even when Keeper is enabled, unlike the sidecar-mode env builder, which withholds it when Keeper is on (seeSECRETdelivery depends on Keeper).
3. MCP Environment Variable Injection
3. MCP Environment Variable Injection
MCP server configurations in
.mcp.json can reference credentials using ${VAR} syntax in their env blocks. Crewship resolves these references at runtime:- The system scans
.mcp.jsonfor${VAR}patterns in env blocks - Each
${VAR}reference is matched against workspace credentials byenv_var_name - Matched credentials are decrypted and injected as actual environment variables
- Claude Code natively expands
${VAR}references from the container env vars
OAuth Token Handling
Credentials with theAI_CLI_TOKEN type or values starting with sk-ant-oat receive special handling:
- The credential is injected as
CLAUDE_CODE_OAUTH_TOKEN(notANTHROPIC_API_KEY) - OAuth tokens use an HTTPS CONNECT tunnel through the sidecar proxy, not the API key header injection path
- This distinction ensures that Claude Code authenticates via OAuth flow rather than direct API key authentication
The sidecar detects OAuth tokens by their
sk-ant-oat prefix and routes them through the CONNECT tunnel handler instead of the standard reverse proxy path.Credential Health Status
Every LLM credential carries a health status that the dashboard, the CLI (crewship credential list) and the runtime token pool all read:
What sets EXPIRED
- The credential health monitor. It polls each provider’s model-listing
endpoint with the stored credential. A
401maps toEXPIRED, a403toREVOKED. Network failures and5xxmap toERROR, never toEXPIRED— a flaky network must not look like a dead key. - The OAuth refresh worker, when a
OAUTH2-type credential’s refresh grant fails (you also get acredential.expiredworkspace event). - An explicit status update written through
PATCH /api/v1/internal/credentials/{id}. Note this is an internal route: its only caller today is the health monitor itself, so there is no operator or sidecar path that moves a credential back toACTIVE.
What makes a credential recover
EXPIRED is not self-healing, and this is deliberate:
-
API keys (
API_KEY) recover on their own once the provider accepts them again — the next health check gets a200and flips the status back toACTIVE. -
Claude Code CLI / OAuth tokens (
AI_CLI_TOKEN, values startingsk-ant-oat) do not. There is no endpoint that can validate one, so the monitor never probes them — and therefore never asserts they are healthy either. Ansk-ant-oatcredential that has goneEXPIREDstaysEXPIREDuntil you replace the token:
Only
ACTIVE credentials are ever decrypted for a run. An EXPIRED or
REVOKED credential still appears in listings (so you can see and fix it)
but its value is never injected.Monitoring Credential Exposure
Some credentials unavoidably land in the agent’s environment — OAuth tokens and CLI tokens use a CONNECT tunnel / env-var read that the proxy can’t isolate, and SECRET-level credentials are exposed when Keeper is disabled. Whenever a credential does land in the agent env, the Orchestrator logs the exposure so operators can see it and remediate where possible:- WARN (actionable) — a SECRET credential exposed because Keeper is off.
Enable Keeper (
KEEPER_MODEL/KEEPER_OLLAMA_URL) to gate it behind/keeper/requestinstead. - DEBUG (informational) — OAuth/CLI tokens that are structurally un-isolatable. Logged for awareness, not remediation.
crewship agent get <slug>(debug output /service_logs)GET /debug/logs?level=WARN&agent_id=<id>for programmatic access
Master Encryption Key Operations
All credential values are encrypted at rest with AES-256-GCM under a single instance-wide master key (ENCRYPTION_KEY). Two operator concerns follow
from that:
Where does the key live?
On first bootcrewship start auto-generates the key and persists it to
<dataDir>/secrets.env — on default installs the same volume as the
database, so a copied disk carries both the ciphertext and the key. The
server warns about this at startup on every boot, and reports it on the
health surface:
ENCRYPTION_KEY from an external secret store and remove
the entry from secrets.env.
Rotating the master key
crewship admin reencrypt re-encrypts every stored envelope — credential
values, refresh tokens, OAuth client secrets, webhook signing secrets,
Composio keys, PKCE verifiers — to the current key
version, so the old key can be retired instead of living in the environment
forever:
What’s Next
Keeper Security
Set up AI-powered credential access control with security levels L1-L4.
Encryption Details
Deep dive into AES-256-GCM encryption, byte layout, and key versioning.
Orchestration
Create multi-agent missions with credential failover and CooldownManager.
CLI Reference
Complete CLI reference for credential management commands.