Skip to main content

​
Credentials

Crewship encrypts all credentials at rest with AES-256-GCM and delivers them to agents through a sidecar proxy — never as environment variables.

​
Credential Types

Canonical list in internal/api (CredentialType constants). The manifest validator accepts any of these strings on a credentials[].type: field; UI and CLI use the same set.

​
Custom Fields (multi-part credentials)

A credential’s own value is a single secret. Plenty of credentials are not one secret: AWS static credentials are an access key id, a secret access key and a region; a service-account credential is a blob plus a filename; a login may carry a TOTP seed, a passphrase, a host or an account id. Rather than a column per shape, a credential can carry custom fields — any number of named parts, each of which is either secret or plain: Plain fields are cleartext on purpose, for the same reason credentials.username is: an identifier is not a secret, and storing it in the clear lets the UI search and sort without decrypting every row while keeping the AES-GCM surface small. There is no way to read a secret field’s value back — not through the API, not through the CLI. crewship credential reveal is the single disclosure path and it applies the full reveal ceremony; rotating is usually what you actually wanted (see Rotation Permissions).
set is an upsert: it creates the field, and replaces the value if the key is already there. Omitting --plain stores the field encrypted — the default is the safe one, because a field whose secrecy you forgot to state is exactly the one you cannot afford to publish.

​
Rules

  • Key shape — lower_snake_case, starting with a letter, at most 64 characters (^[a-z][a-z0-9_]{0,63}$). Lowercase-only because a key becomes an environment-variable name and a file name at delivery time, where Region and region would collide once either is upcased.
  • Reserved keys — value, password and username are refused. Those live on the credential itself, set through the credential’s own value rather than a field; a second copy under a field would drift away from the first with nothing to reconcile them.
  • Limits — at most 32 fields per credential, each value at most 64 KiB (the same cap as the credential’s own value).
  • Duplicates — one key per credential. POST answers 409; the CLI’s set turns that into an update.
  • Visibility and role — reading fields uses the same crew-scoped visibility rule as reading the credential, and writing them requires the same role as PATCH /api/v1/credentials/{id} (MANAGER and above). A field is never a way around the gate on the credential it belongs to.

​
How fields reach the agent

Fields are delivered with the credential they belong to, under a name derived from the credential’s own environment variable:
SLOT is whatever the credential resolved to on this agent — the env var of an explicit assignment, the slot of a binding (see “Slots and Bindings” below), or (when neither exists) the credential’s name. It is never derived from the field. That is what keeps ten accounts of one provider apart: bind GH_TOKEN to one GitHub account and GH_ACME to another, and their account_id fields arrive as GH_TOKEN_ACCOUNT_ID and GH_ACME_ACCOUNT_ID, each next to the token it describes. For the AWS credential above, assigned to the agent as AWS:
A field never overwrites a variable something else already claimed. If the derived name is taken — by another credential’s slot, by another field, or by a name the runtime owns (HOME, PATH, HTTP_PROXY, CREWSHIP_*, CLAUDE_CODE_OAUTH_TOKEN, the provider base URLs) — the field is dropped, the variable keeps the value that was already there, and the server logs a warning naming the credential, the field key, the derived name and the reason. A field hangs off the slot its credential is delivered under, not the raw name — a credential called github-acme arrives as GITHUB_ACME, so its region field arrives as GITHUB_ACME_REGION. A credential whose name yields no variable at all is not delivered, and neither are its fields: there is no prefix left to name them with. Dropping is the deliberate choice: a missing AWS_REGION is one env away from being diagnosed, while an AWS_REGION holding some other credential’s value is not diagnosable at all — and the agent will act on it. Beyond the naming, fields follow their credential exactly:
  • Which channel — a file-delivered credential (see below) gets one 0400 file per field under /secrets/<agent>/, with .env mapping the name to the path, the same way USERPASS already produces <envvar>_USERNAME and <envvar>_PASSWORD.
  • Secret fields are decrypted at delivery through the same path as the credential’s own value, and they reach the agent’s environment only where that value does. Under Keeper a gated credential’s secret fields are withheld with it.
  • Plain fields are cleartext from storage to container. They are identifiers, not credential material, so they are delivered even when the credential’s own value is isolated behind the sidecar proxy — a proxy can inject a key into a request, it cannot tell your CLI which region to use.
  • Revocation — a soft-deleted, revoked or expired-lease credential delivers no fields either. There is no path by which a part outlives the whole.

​
Supported Providers

Credentials are associated with a provider for automatic injection:

​
How Credential Injection Works

Agents never see raw API keys. The sidecar proxy intercepts outbound HTTP requests and injects credentials based on the destination host:
The sidecar also supports a reverse proxy mode where agents point a provider base URL at the sidecar and send requests to it directly over plain HTTP; the sidecar swaps the dummy key for the real one and forwards to the upstream. Three providers use this today:
  • Claude Code — ANTHROPIC_BASE_URL=http://127.0.0.1:9119 → api.anthropic.com.
  • Codex — OPENAI_BASE_URL=http://127.0.0.1:9119/openai/v1 → api.openai.com (the /openai prefix keeps it distinct from Anthropic’s /v1/ on the shared port, and is stripped before forwarding).
  • Gemini — GOOGLE_GEMINI_BASE_URL=http://127.0.0.1:9119/gemini → generativelanguage.googleapis.com (the /gemini routing prefix is stripped before forwarding; the real key is injected as the x-goog-api-key header, overwriting the dummy the CLI sends).
For these adapters the real key never enters the agent environment — only a dummy *_API_KEY placeholder does, which the sidecar overwrites mid-flight. The remaining adapters (OpenCode, Cursor, Factory) still read their real key from an env var and reach their upstream over an HTTPS CONNECT tunnel; those keys are visible in the agent process (surfaced by AgentEnvCredentialExposures so the gap is observable, not silent). OpenCode is BYOK-by-design (its driver dials 75+ providers directly), and Cursor/Factory cannot join the reverse-proxy model until their CLIs ship an endpoint override — cursor-agent’s configuration surface has none today.

​
Slots and Bindings — more than one account per provider

A credential’s name is the account (github-acme). The slot is the environment variable the agent reads (GH_TOKEN). They used to be the same string, which meant a workspace could hold exactly one GitHub account: a second one would also have had to be called GH_TOKEN, and credential names are unique per workspace. A binding connects the two:
Scope is workspace, crew or agent.

​
Resolution order

agent > crew > workspace. The most specific scope that binds a slot wins, and only that one is delivered — an agent never receives two candidates for the same env var.
source names the rule that won: agent_grant (an explicit crewship credential assign), agent_binding, crew_binding, workspace_binding, or crew_link — no binding at all, delivered under the credential’s own name.
Credentials with no binding are unchanged. A crew-linked credential named CREW_TOKEN still arrives as CREW_TOKEN, and an explicit per-agent assignment still uses the env_var_name you chose. Bindings are additive: nothing is renamed, and no existing workspace has to adopt them.
An environment variable name is uppercase letters, digits and underscores, and does not start with a digit — GH_TOKEN, _PRIVATE, AWS_KEY_2. That is the rule the container enforces, so it is the rule everything here agrees on. Slots and env_var_name are variables, so they are held to it at write time, and stored in the form the agent will see. You may type a dash or lowercase; the server records the uppercase form and the response shows it:
A name no variable can be derived from (gh token, 2fa, anything non-ASCII) is a 400 on the request that chose it. A credential’s name is not a variable — it is the account (github-acme), and it stays whatever you called it. It only becomes a variable when there is no binding and no assignment, in which case the credential is delivered under its own name. When that name is not already a legal variable, it is normalised on the way in — github-token arrives as GITHUB_TOKEN — and crewship credential resolve tells you so:
Two cases are not delivered, and each is reported the same way:
  • The name folds onto one that is taken. github-token and GITHUB_TOKEN are two different credentials (names are unique per workspace) that want one variable. The one that asked for GITHUB_TOKEN by name keeps it; the other is not delivered. Handing an agent a variable that plausibly holds a different account’s token is the failure this avoids — a missing GH_TOKEN is diagnosable in one command, the wrong GH_TOKEN is not.
  • No variable can be derived at all (my token!). Nothing is invented for it.
One badly-named credential never costs the others, or the run. Before this, a credential named github-token failed the variable check while its files were being written, and that failure abandoned the whole credential batch — an agent holding an SSH key and a CLI token alongside it did not start at all, and the error blamed the Docker daemon’s version. A credential that cannot be named is now skipped on its own, with a warning naming it.

​
One slot, one credential, per scope

Within a single scope a slot points at exactly one credential. Binding a slot that is already taken in that scope returns 409 Conflict rather than replacing the existing row — silently repointing every agent in a crew at a different account is not something a write should do implicitly. Unbind first, then bind. Binding the same slot in different scopes is not a conflict; that is the whole point.

​
The honest limit

A container has one default identity per tool. gh reads GH_TOKEN and nothing else. Ten crews with ten accounts is fine. One crew that needs two GitHub accounts at once hits the limit: the second account cannot be the default and must get an explicit slot.
The second one only helps where the tool itself takes an explicit choice (gh --hostname, per-remote git config, AWS_PROFILE).

​
Deleting the scope deletes the binding

Bindings cascade with their owner: deleting a crew removes its crew-scoped bindings, deleting an agent removes its agent-scoped ones, and deleting a credential removes every binding pointing at it. A binding cannot outlive the thing it was scoped to and keep claiming a slot.

​
Priority-Based Selection

The CredStore (internal/sidecar/credstore.go) selects credentials using a two-tier system:
  1. Priority tier: Credentials with the lowest numeric priority value are selected first (lower = higher priority)
  2. Round-robin within tier: Multiple credentials at the same priority level rotate to distribute load
The CredStore is an in-memory store — credentials are never written to disk inside the container. They are loaded at container startup via stdin JSON from the orchestrator.

​
Credential Delivery Flow

1

Encrypted storage

Credentials are stored in the database encrypted with AES-256-GCM. The format is v1:base64(IV||AuthTag||Ciphertext) where IV is 16 bytes and AuthTag is 16 bytes.
2

Decryption at runtime

When a crew container starts, the orchestrator decrypts assigned credentials using the ENCRYPTION_KEY environment variable.
3

Piped via stdin

Decrypted credentials are sent to the sidecar process as JSON via stdin — not environment variables. This prevents credential leakage through /proc/environ or ps aux.
4

In-memory CredStore

The sidecar loads credentials into the CredStore (a thread-safe in-memory map). The CredStore supports concurrent access with sync.RWMutex.
5

Automatic injection

When the agent makes an HTTP request, the proxy matches the destination host to a provider and injects the appropriate authentication header.

​
Cooldown Management

When a credential receives a 429 Too Many Requests response, the CooldownManager (internal/orchestrator/) temporarily removes it from the rotation. This prevents thundering-herd problems when rate limits are hit.

​
Credential Security Levels

Credentials can be tagged with security levels for Keeper gating:
L1 auto-allow never applies to /keeper/execute requests. Commands must always be evaluated by the Keeper LLM to prevent exfiltration attacks like echo $TOKEN | base64.

​
Where the tier shows up in the console

The tier is the property that decides what happens when an agent asks for the secret, so it is on every credential surface rather than only inside the edit form:
  • An L1–L4 chip beside the name — in the left rail, and on both of the overview’s lists (Needs attention and Recently used). A credential whose server did not report a tier shows L?, never L1: “we were not told” and “unguarded” are different claims. The credential’s own page carries the same fact at full width instead, as a toned pill in its header — reading Tier not reported where the chip would read L?.
  • A Tier section in the left rail — single-select, one row per tier, with the workspace’s count on each. Unlike every other facet on that rail, empty tiers stay on the list: L4 · critical — 0 is the answer to “does anything here stop for a human?”, not a dead control.
  • A Security tiers donut on the overview, whose arcs sum to the vault. Clicking an arc selects that tier in the rail.
  • A Guarded · L3+ tile, counting the credentials Keeper mediates per read rather than handing to the agent for the whole run.
A stored level the tier table does not define is read as L4 everywhere in the console, matching keeper.SecurityLevel.Tier() — unknown blast radius is treated as the strictest tier, not the loosest.

​
One credential, one page

Opening a credential replaces the dashboard with that credential in full, laid out exactly like an issue: an identity card, a band of figures, then the substance in a wide column beside its properties in a narrow one. It stacks to a single column on a phone, where the rail collapses to a stub and opens as an overlay. It used to be five tabs — Overview, Fields, Used by, Audit, Settings. Tabs are a way of admitting a screen holds more than fits, and this one does not: the value, its parts, who can read it, how hard Keeper guards it and what has happened to it are the five things you weigh together when deciding whether to rotate something. Everything is on the page at once, with the audit log last. One secret, one way to replace it. The Value card offers rotate — which mints a new value and keeps the old one working through the grace window — and nothing else. A plain swap with no overlap is a property of the credential like any other, so it lives in Edit: leave the Value field empty there to keep the existing secret. Edit is also where the credential’s icon is chosen; the brand picker beside the name is what the rail, the dashboard and this page all draw. The audit says who. Every row carries the actor: an agent with the avatar it has everywhere else in the product, a person with their name, or system for a row nobody signed. The actor used to be recorded in two different shapes — the agent_id column on the sidecar path, and one of revealed_by / rotated_by / created_by / approved_by / rejected_by in metadata on the human ones — so the timeline could say what happened and not who did it. GET /credentials/{id}/audit now resolves all of them into actor_kind, actor_id and actor_name. Ten events are shown at a time with the total beside them (10 of 37); a credential read every few minutes is otherwise fifty rows of “USE · 3m ago”. crewship credential audit <name> prints the same two columns (ACTOR, WHO). A sidecar fetch is the one event with no agent behind it — a sidecar serves a whole container, not one agent — so it is attributed to the crew that owns it. A sidecar polling workspace-wide has no crew either, and those rows say system, which is the honest answer rather than a guess. Reveal says why it is unavailable. It has four gates — the workspace switch, the MANAGER floor, the credentials:reveal capability, and the SEALED classification — and failing any of them used to render nothing at all. Each gate is a different fix, so the card names the one that is shut. Audit and rotation history are still gated on role, not on layout: GET /audit is MANAGER+ because it exposes the IPs behind admin actions, and rendering an empty timeline for someone who is merely not allowed to read it would be a false statement rather than a missing section.

​
The Credentials overview

Opening Credentials with nothing selected shows the vault’s dashboard, not a table: what is active, what is guarded, what is missing a CLI, what expires inside 30 days; then the tier donut, the queue of what needs attention, the breakdown by type, and either what expires next or what was read most recently. Picking a credential in the rail replaces it with that credential’s detail. There is no credential table. The left rail is the list — searchable, filtered, sorted, and multi-selectable — so the main pane never shows the same names a second time. What the old table’s columns carried moved to where each fact belongs: The cards report the whole workspace on purpose; the rail’s filters narrow the rail, not the dashboard. A donut that redrew itself to one slice the moment you clicked a slice would answer nothing.

​
Narrowing the list

The rail’s search box matches a credential’s name, account label, description, tags and the names of the agents holding it. Everything else is behind the Filter button, and each group draws its rows with the thing they are about rather than one repeated glyph:
  • Category — the brand marks actually inside that category, commonest first.
  • Scope — a workspace icon, then each crew’s own tile and colour.
  • Assigned to — the agents holding at least one credential, each with the same avatar it has everywhere else in the product. Selecting one answers “what can this agent read?”.
  • Tag — every tag in use, with the number of credentials carrying it.
Status and tier stay in the rail itself rather than in the dropdown: both are single-select, both are bounded, and both are asked on arrival. Selecting several credentials is a mode, not the rail’s resting state: press Select in the Credentials header and the rows grow checkboxes; press Done (or Cancel on the floating bar) and the selection goes with it. A checkbox on every row all the time says the list is a thing you tick, when it is overwhelmingly a thing you click — and it leaves a bulk delete one mis-click from every secret in the vault.

​
Listing & Filtering Credentials

crewship credential list returns the workspace’s credentials. For large workspaces it supports server-side filtering and cursor pagination:
--search and --tag are applied on the server. When more results remain and you didn’t pass --all, the CLI prints the cursor for the next page to stderr; pass it back with --cursor <value>. Under the hood these map to the GET /api/v1/credentials query parameters search, tag, limit, and cursor. The endpoint is backward compatible: without paginate=true it returns the plain array it always has; with it, the response is a { "credentials": [...], "next_cursor": ... } envelope (next_cursor is null on the last page). The web Credentials page follows the workspace selected in the top-bar switcher, so users in multiple workspaces can manage each one’s credentials. It has no table — the left rail is the list; see One credential, one page.

​
Adding Credentials

​
Via the UI

Open Credentials and click Add secret. The flow asks three questions, in this order:
  1. What shape is it — Token, Login, Key pair, SSH key, File or Certificate. The shape decides which boxes you fill; there is no brand catalog to hunt through. Any of the thousands of services out there fits one of these six, and anything else fits a custom field.
  2. The values the shape implies, plus a name for the account (github-acme, not GH_TOKEN — see Slots and Bindings). Parts marked plain in the table above are stored in the clear; everything else is encrypted and never readable again.
  3. Who gets it, and under which variable — the whole workspace or selected crews, then the slot: the environment-variable name the container will see. Leave the slot empty and the credential is delivered under its own name, which is the pre-slot behaviour.
Pasting a recognised secret (ghp_…, sk-ant-…, AKIA…) picks up the brand icon and suggests a slot name. That is a hint, not a gate — an unrecognised value is completely normal and nothing blocks on it. Choosing the slot requires workspace-admin rights (POST /api/v1/credentials/bindings is OWNER/ADMIN). A MANAGER can still create the credential; an admin can bind it to a slot afterwards. Secret values (and OAuth refresh tokens) are capped at 64 KiB per value; ENDPOINT_URL values are capped at 2048 bytes. Oversized values are rejected with a 400 at submit time. A credential may carry at most 32 custom fields.

​
The Credentials list

The list has a left rail in the same shape as Integrations: Status (everything / needs attention / missing tool), Category, Scope and Tag, each with its own count. The Readiness column is the one that is not about the vault. It answers “does the crew’s container actually have the CLI this credential is for?”, from GET /api/v1/crews/{crewId}/credential-readiness, and it has three states — not two:

​
Revealing a value from the UI

The credential detail sheet offers Rotate and show the new value as the primary action and Reveal the existing value… as the secondary one. That ordering is deliberate: most reasons to want a value are really reasons to replace it, and a control that is used rarely is a control that keeps working. The Reveal action only appears when all of the following hold, which is exactly what the server checks:
  • reveal is enabled for the workspace (Settings → Access & Secrets),
  • your membership holds the credentials:reveal capability — being an OWNER is not sufficient, it is granted per person and belongs to no role bundle,
  • your role is MANAGER or above, and
  • the credential is not SEALED.
The reveal dialog itself requires a written reason of at least 20 characters, records it in the tamper-evident journal before the value is returned, and shows the value once. Closing the dialog discards it; a second look needs a second reveal, with its own audit entry.

​
Settings → Access & Secrets

Workspace-wide policy lives in Settings → Access & Secrets, not on any one credential’s page:
  • the reveal switch — read by MANAGER and above, changed by the OWNER only (an ADMIN is refused by the API, so the switch renders read-only for them and says why),
  • who holds credentials:reveal, with a warning once it is spread wider than a couple of people,
  • what each classification means and who may move a credential between them (MANAGER+ to raise, OWNER/ADMIN to lower — lowering is journaled).
Per-category default classifications are not configurable yet; set the class on each credential from its detail sheet.

​
Via the CLI

--crews accepts crew slugs or IDs and is the CLI equivalent of the UI’s crew-scoping picker (it writes the credential_crews junction). Supplying crews sets the credential’s scope to CREW; omit it for a workspace-wide credential. --scope WORKSPACE|CREW is available to set the scope explicitly, but it’s normally inferred from --crews. --scope only accepts WORKSPACE or CREW (case-insensitive) and is rejected client-side otherwise; on credential update it also rejects an empty value. Note that the server always derives scope=CREW when --crews resolves to a non-empty list, even if --scope WORKSPACE was passed explicitly — the CLI prints a warning to stderr in that combination, but the server’s derived scope wins.

​
Via Seed Data

For development, set environment variables before seeding:
The seeder auto-detects OAuth tokens (prefix sk-ant-oat) vs API keys and creates the appropriate credential type.

​
Rotation Permissions

Rotating a credential (and cancelling a rotation’s grace window early) requires the OWNER/ADMIN role or the credential.rotate member capability. Granting that capability to a MANAGER or MEMBER lets an oncall user rotate a leaked token without giving them blanket vault access — see the capability list in the Workspaces API reference.

​
Credential Assignment

Credentials are assigned at the workspace level and automatically distributed to agents based on provider matching. Two assignment modes exist:

​
Short-lived leases (TTL)

An assignment is normally a standing grant — long-lived, reused across every session until it is unassigned. A stolen standing grant stays valuable indefinitely. A lease flips that: the grant is issued with a short TTL, and once it lapses it is refused at credential-injection time (fail-closed) — the agent must be re-granted access. The durable secret always stays in the vault; the grant is the only thing that expires. Add a lease by passing --ttl to credential assign:
--ttl accepts Go durations (30m, 2h, 24h), capped at 30 days — a multi-month “lease” is a standing grant in disguise and is rejected. Omit --ttl for a standing grant. Enforcement is at the injection point, not just the UI, and it covers every path that hands a grant’s plaintext to an agent: Server-side gates decide what to deliver at the instant of delivery. A container then holds that plaintext for its whole life, so leases also travel with the delivered credential:
  • The boot payload carries each credential’s lease_expires_at.
  • The sidecar’s credential store refuses a credential whose lease has lapsed the moment it lapses (not on the next sweep), and the credential reaper evicts it from memory within one ~60s interval so the plaintext stops being resident.
  • Unlike revocation — which needs a crewshipd round-trip and therefore fails open, so a transient blip cannot nuke a working key — lease expiry needs no round-trip and therefore fails closed. An unreachable server is not an excuse to keep serving a lapsed lease.
A captured lease is unusable after its TTL, whichever path would inject it.
The lease lives on the per-agent grant. A credential whose scope is WORKSPACE (or that is granted to a crew via credential_crews) is still reachable through that path, which carries no TTL — a lease narrows one agent’s grant, it does not retro-scope a workspace-wide credential. Scope the credential to the agents that need it if you want the lease to be the only way in.
Lease state and provenance are visible on the agent’s credential list:
An expired lease renders as EXPIRED <timestamp>. SOURCE says which event minted it — see the next section.

​
Auto-issued leases (on approval)

--ttl leases one grant by hand. Auto-lease does it for you: with a TTL configured for the workspace, every approval re-issues the grant as a lease, so credential access decays instead of accumulating.
Once set, a lease is minted (SOURCE in parentheses) when:
  • the Keeper ALLOWs a /keeper/request or /keeper/execute call (keeper_allow), or
  • a human approves an agent-proposed CREDENTIAL escalation (escalation_approve).
Each approval refreshes the lease, so an agent that keeps working keeps its access — and one that stops asking loses it after the TTL. Four rails keep this safe to turn on:
  • Opt-in. Default off. Nothing changes until you set a TTL.
  • L3/L4 only. L1/L2 self-service credentials are delivered to the agent for the whole run (they are how it calls its own model), so they are never auto-leased — expiring one mid-run would break the agent’s own work rather than contain an attacker.
  • Never shortens a longer lease. A grant you leased with --ttl 7d keeps its 7 days; only a standing grant or a shorter lease is rewritten.
  • Bounded. Minimum 60s (a shorter lease can lapse inside Keeper’s own evaluation, denying the request that authorised it), maximum 30 days.
Every mint writes a LEASED row to the credential’s audit timeline and a credential.lease_issued entry to the journal, carrying the source, the resulting expiry and the authorising request id — so “why did this credential stop working at 14:32?” is answerable.
Turning auto-lease off does not un-lease grants that are already leased — they keep their expiry. Extending live leases on a config change would be the wrong direction for a security control. Re-grant without --ttl to make one standing again.
With auto-lease on, approving an agent-proposed credential escalation also creates the proposing agent’s grant (leased) if it had none. With auto-lease off, approve behaves as it always has: it activates the credential and creates no grant.

​
Agent-Proposed Credentials (pending approval)

Agents cannot activate a credential on their own — the lifecycle requires a named human, and the /secrets/{agent-slug}/ files carry no authority — they are plain 0400 copies on the crew tmpfs, and rewriting one does not register anything in the vault. But an agent that generated a secret for the crew — say a password for a database it just provisioned — can propose it, and a human approves it with one click. The agent raises a CREDENTIAL escalation through the sidecar, carrying the proposed credential as JSON in the metadata field. Send the body over stdin (--data @-) so the secret never lands in the shell history or process arguments:
Attribution comes from the per-agent token, not the from field. Each agent is handed its own bearer token ($CREWSHIP_AGENT_TOKEN, derived HMAC(master, workspace‖agent)) in its environment and injected into its MCP config. The shared per-crew sidecar matches that token against the crew roster to resolve the acting agent: a valid token is authoritative and overrides any from in the body — so a sibling sharing the container can no longer POST from=<peer> to steal attribution — and a token that matches no crew member is refused. When no token is presented (legacy callers) the sidecar falls back to validating from against crew membership. This closes the intra-crew impersonation gap for escalations, peer queries, the memory path, and Keeper credential requests.
What happens:
  1. The value is encrypted and stored immediately as a credential with status PENDING_APPROVAL. It shows up in the Credentials page (amber Pending approval badge, under Needs attention) and as an item in the inbox, but it is never delivered to any agent while pending.
  2. A human Approves — from the inbox (one click), the crew escalations panel, or crewship escalation resolve <id> --action approve — and the credential flips to ACTIVE, attributed to the approver. Only now can the crew use it.
  3. Or Rejects (--action reject) and the proposed credential is discarded.
The agent’s /escalate call blocks until the human approves or rejects (up to 5 minutes). If the agent does not have the value itself and needs a human to supply it, it omits metadata and describes the need in context instead (the legacy human-supplies-the-secret flow). Provenance is preserved either way: the credential records the agent as proposer and the human as approver.

​
When a proposal can’t be staged

The proposed secret is discarded (never persisted in the clear) the moment it is read, so the escalation never reflects success it didn’t achieve:
  • Name already in use or unknown credential type — recoverable. No credential is staged, but a plain escalation is still raised with a note explaining why, so a human can supply or rotate the value manually. The call returns 201.
  • No workspace owner to approve the proposal, or a vault/encryption error — a hard failure. No escalation is recorded and the call returns 503 with a message the agent can act on (retry, or ask an operator to configure a workspace owner). This avoids a phantom “pending approval” that no one can act on while the secret is already gone.

​
Segregation of duties (second approver)

By default, any workspace MANAGER+ can resolve a CREDENTIAL escalation below L4 — including one raised by an agent they created themselves. Regulated workspaces can opt in to a strict four-eyes rule for every tier: the person who proposes a credential (via their agent) can never also be the one who approves it.
L4 · critical credentials always require a second approver, whether or not the workspace opts in — the tier forces the rule and the toggle can only tighten it, never loosen it. See Keeper → The switch is a floor, not a master switch for the full table and for where the console and CLI report which of the two applies.
Enable it per workspace (requires OWNER or ADMIN):
When enabled, crewship escalation resolve (and the inbox Approve/Reject button) is refused with a 403 for the user recorded as the owner of the agent that raised the CREDENTIAL escalation (agents.created_by_user_id — whoever created that agent). This is checked for every resolution action (approve, reject, redirect), not just approve, and before any row is mutated. Every blocked attempt is written to the audit journal (a keeper.decision entry naming the segregation_of_duties rule and the blocked user).
OWNER is not exempt. This is a strict segregation-of-duties gate, not a permission check — even a workspace OWNER is refused if they are also the recorded owner of the agent that raised the escalation. Someone else with MANAGER+ role must resolve it instead. The rule only applies to CREDENTIAL escalations; TEXT/LINK escalations are unaffected. If the raising agent has no recorded owner (a legacy agent created before this attribution existed), the rule can’t be enforced and resolution proceeds as before.
Scope limit — identity is agent ownership, not the driver. “Initiator” is the agent’s recorded owner (created_by_user_id), not necessarily the human who drove the agent to raise the escalation. If user A owns an agent but user B drives it (via chat) to propose a credential, the rule blocks A — user B could still self-approve. This covers the common case (owner drives their own agent); tightening it to the actual requester requires recording the driving user on the escalation at raise time (planned follow-up).
Two guard rails. Enabling the rule on a workspace with fewer than two members who can approve (OWNER/ADMIN/MANAGER) still succeeds but returns a warning — with only one eligible approver a credential raised via their agent could never be resolved by someone else. And crewship credential rotate (POST /credentials/{id}/rotate) is refused with a 409 for a credential still PENDING_APPROVAL, so rotation can’t be used to activate an agent-proposed credential outside the four-eyes flow — approve or reject it via its escalation.
The toggle rides on the same per-workspace governance row as the Keeper watchdog settings (GET/PUT /api/v1/admin/keeper/governance) — it is a distinct concern (who may approve a credential escalation, not behavior monitoring) but reuses the same OWNER/ADMIN-gated, journal-audited settings surface. Default is off.

​
File-Based Secrets

For non-LLM credentials (database passwords, API tokens), Crewship writes one file per credential to /secrets/{agent-slug}/:
The agent reads credentials from these files. The .env file maps environment variable names to file paths for tooling compatibility.

​
Lifecycle: in-memory, per-run, removed on completion

/secrets is an in-memory tmpfs mount (16 MiB, mode 0700, owned by the agent UID) — never a host directory. Credential files exist only in RAM inside the running container: they never touch the host disk, never survive a container stop, and can’t leak into host backups or archives. Requires Docker Engine 26+ for the tmpfs ownership options; crew containers created by older Crewship versions (which used a host bind mount) are automatically recreated with the tmpfs on their next use, and the legacy host-side directory is deleted. The files are also per-run: Crewship rewrites them at the start of every agent run and removes /secrets/{agent-slug}/ again when the run finishes (refcounted, so overlapping runs of the same agent are safe). Between runs there is nothing under /secrets/{agent-slug}/ for another container process to read.

​
SECRET delivery depends on Keeper

SECRET credentials follow the Keeper state, and the file path mirrors the env-var path exactly:
  • Keeper disabled — the SECRET is delivered as a 0700/0400 file under /secrets/{agent-slug}/ (and mapped in .env), just like CLI_TOKEN and GENERIC_SECRET. This is the legacy default and is unchanged.
  • Keeper enabled — the SECRET is not written to the agent’s filesystem at all, and it is not injected as an environment variable. The agent’s system prompt states it does not have the value in its environment; to use it the agent must request it through the Keeper API (/keeper/request for the raw value, or /keeper/execute to run a command with the secret injected), which enforces access control and writes an audit trail. Withholding happens once, at the config resolver — the plaintext is blanked before it ever leaves the server process, so the env, file, and MCP delivery paths gate on Keeper as defense-in-depth rather than as the sole line. The resolver logs each withheld credential at WARN (env-var name only, never the value), so the gate is observable rather than a silent skip. There is no file to read and nothing under /secrets/{agent-slug}/ for a compromised process to scrape.
Which credential types are Keeper-gated is decided by a single per-type delivery-policy table (internal/credpolicy), not by a SECRET-shaped special case scattered across the delivery paths — so a new credential type is governed by one row, and an unclassified type fails safe (withheld and not delivered). Today the gate is SECRET-only: CLI_TOKEN, GENERIC_SECRET, USERPASS, SSH_KEY, and CERTIFICATE are always delivered as files regardless of Keeper state — CLI tools and generic secret consumers read them from disk, and the prompt makes no withhold claim for them.
MCP config references are covered too. The withholding above spans all three credential-delivery paths — the credential files, the direct env builder, and MCP env injection. If a SECRET’s env-var name is referenced by an MCP server config (e.g. Authorization: Bearer ${DB_PASSWORD}), Crewship does not inject the value into that MCP process’s environment when Keeper is enabled: the SECRET is withheld exactly as it is on the file and direct-env paths, so it stays out of /proc/self/environ and out of the /keeper/* audit bypass.A consequence follows: because the MCP process runs inside the agent container and cannot call the Keeper API itself, an MCP server that genuinely needs the value will not receive it while Keeper is gating that SECRET. If a server must authenticate with a credential, give it a non-SECRET credential type (CLI_TOKEN, GENERIC_SECRET, …), or run that server without Keeper-gating the secret it needs. The gate stays SECRET-only and Keeper-only — non-SECRET types referenced by MCP configs are injected as before, whether Keeper is on or off.

​
The trust boundary is the crew, not the agent

Every agent process in a crew container runs as the same principal — UID 1001. The 0700 mode on /secrets/{agent-slug}/ protects it from the sidecar (UID 1002), not from sibling agents: to the kernel, all agents in one crew are the same user, so any agent can read and write every sibling’s /secrets/{slug}/ files, home directory (/crew/agents/{slug}/), output, and MCP configs, and can signal every sibling’s processes. The per-run write/remove lifecycle above shrinks the at-rest window, but while two agents run in parallel there is no intra-crew confidentiality. This is the runtime’s deliberate posture, not an oversight: the enforced isolation boundary is the crew. Each crew gets its own container, its own in-memory /secrets tmpfs, its own home volume, and its own sidecar — a sibling crew can never reach any of it. Within a crew, agents are assumed to trust each other, the same way processes owned by one Unix user trust each other.
Put agents in the same crew only if they may read each other’s credentials and files. If two agents must be mutually isolated — different credential sensitivity, different trust levels, one runs third-party or experimental prompts — put them in separate crews. Crew-scoped credentials (crewship credential update <name> --crews ...) keep a sensitive credential out of every other crew’s container entirely.

​
Revoking a file-based secret

Deleting a credential (crewship credential delete <name>, or the UI) removes its /secrets/{agent-slug}/ file(s) from every running crew container that mounted it — immediately, not just on the next boot. The same removal fires when the sidecar reports a credential’s status as REVOKED (e.g. the upstream provider invalidated it). The removal runs server-side, exec’d inside the container as the agent UID (1001): that’s the only principal that can unlink inside the agent’s 0700 secrets directory, so the sidecar (UID 1002) can’t do it and the server does. A successful delete has already cut off live access. A multi-part credential’s custom field files go with it — the removal covers every file the credential delivered, not just its primary one, so a revoke never leaves a secret access key behind while reporting the credential gone. It’s best-effort: a stopped container is skipped (nothing to remove; the revoked secret won’t be re-written on the next boot because it’s filtered from every delivery path). The .env map keeps an inert reference to the removed file until the container next boots — harmless, since the file is gone. If you need a guaranteed clean slate, recreate the crew container.

​
Three Credential Injection Modes

Crewship supports three distinct modes for delivering credentials to agents, depending on the container configuration and credential type.
The standard and most secure mode. Credentials are piped to the sidecar process via stdin as JSON at container startup. The sidecar holds them in an in-memory CredStore and intercepts outbound HTTP requests, injecting the appropriate authentication headers based on the destination host.
  • Credentials never appear in environment variables or on disk
  • The sidecar runs as UID 1002, inaccessible to the agent (UID 1001)
  • Supports priority-based selection and round-robin rotation
Privileged crews are the exception. A crew provisioned with a devcontainer feature that requires --privileged (e.g. Docker-in-Docker) drops no-new-privileges and CapDrop:ALL for the whole container — the same UID 1001/1002 boundary that isolates the sidecar’s memory (and the credentials loaded into it) from the agent no longer holds. A compromised or prompt-injected agent in a privileged container can read the sidecar’s process memory directly.Crewship fails closed by default: the agent-config resolver omits credentials entirely for a privileged crew, and provisioning logs a WARN (#1032) so the trust downgrade is visible in ops. To keep credential injection working for a privileged crew anyway — accepting that the isolation boundary is gone — a workspace OWNER or ADMIN opts in explicitly:
This is a workspace-wide flag, not per-crew: it affects every privileged crew in the workspace. Turn it off the same way (--allow-privileged-credentials=false) once the crew no longer needs it.
When the sidecar is disabled, credentials are injected as environment variables using BuildEnvVars(). This mode is less secure because credentials are visible in the process environment.
  • API keys are set as ANTHROPIC_API_KEY, OPENAI_API_KEY, etc.
  • AI_CLI_TOKEN credentials with sk-ant-oat prefix are mapped to CLAUDE_CODE_OAUTH_TOKEN
  • Secret-type credentials are excluded from environment variables in sidecar mode
MCP server configurations in .mcp.json can reference credentials using ${VAR} syntax in their env blocks. Crewship resolves these references at runtime:
  1. The system scans .mcp.json for ${VAR} patterns in env blocks
  2. Each ${VAR} reference is matched against workspace credentials by env_var_name
  3. Matched credentials are decrypted and injected as actual environment variables
  4. Claude Code natively expands ${VAR} references from the container env vars

​
OAuth Token Handling

Credentials with the AI_CLI_TOKEN type or values starting with sk-ant-oat receive special handling:
  • The credential is injected as CLAUDE_CODE_OAUTH_TOKEN (not ANTHROPIC_API_KEY)
  • OAuth tokens use an HTTPS CONNECT tunnel through the sidecar proxy, not the API key header injection path
  • This distinction ensures that Claude Code authenticates via OAuth flow rather than direct API key authentication
The sidecar detects OAuth tokens by their sk-ant-oat prefix and routes them through the CONNECT tunnel handler instead of the standard reverse proxy path.

​
Credential Health Status

Every LLM credential carries a health status that the dashboard, the CLI (crewship credential list) and the runtime token pool all read:

​
What sets EXPIRED

  • The credential health monitor. It polls each provider’s model-listing endpoint with the stored credential. A 401 maps to EXPIRED, a 403 to REVOKED. Network failures and 5xx map to ERROR, never to EXPIRED — a flaky network must not look like a dead key.
  • The OAuth refresh worker, when a OAUTH2-type credential’s refresh grant fails (you also get a credential.expired workspace event).
  • An explicit status update written through PATCH /api/v1/internal/credentials/{id}. Note this is an internal route: its only caller today is the health monitor itself, so there is no operator or sidecar path that moves a credential back to ACTIVE.

​
What makes a credential recover

EXPIRED is not self-healing, and this is deliberate:
  • API keys (API_KEY) recover on their own once the provider accepts them again — the next health check gets a 200 and flips the status back to ACTIVE.
  • Claude Code CLI / OAuth tokens (AI_CLI_TOKEN, values starting sk-ant-oat) do not. There is no endpoint that can validate one, so the monitor never probes them — and therefore never asserts they are healthy either. An sk-ant-oat credential that has gone EXPIRED stays EXPIRED until you replace the token:
Earlier builds flipped an EXPIRED sk-ant-oat credential back to ACTIVE on every monitor tick and persisted that status, on the theory that an un-validatable credential had probably been marked expired in error. The effect was that a revoked token kept being advertised as healthy and handed to agents. The monitor no longer changes the status of a credential it cannot validate — if you see EXPIRED on a CLI token now, it is a real verdict and re-linking the token is the fix.
Only ACTIVE credentials are ever decrypted for a run. An EXPIRED or REVOKED credential still appears in listings (so you can see and fix it) but its value is never injected.

​
Monitoring Credential Exposure

Some credentials unavoidably land in the agent’s environment — OAuth tokens and CLI tokens use a CONNECT tunnel / env-var read that the proxy can’t isolate, and SECRET-level credentials are exposed when Keeper is disabled. Whenever a credential does land in the agent env, the Orchestrator logs the exposure so operators can see it and remediate where possible:
  • WARN (actionable) — a SECRET credential exposed because Keeper is off. Enable Keeper (KEEPER_MODEL / KEEPER_OLLAMA_URL) to gate it behind /keeper/request instead.
  • DEBUG (informational) — OAuth/CLI tokens that are structurally un-isolatable. Logged for awareness, not remediation.
Where to find these logs:
  • crewship agent get <slug> (debug output / service_logs)
  • GET /debug/logs?level=WARN&agent_id=<id> for programmatic access

​
Master Encryption Key Operations

All credential values are encrypted at rest with AES-256-GCM under a single instance-wide master key (ENCRYPTION_KEY). Two operator concerns follow from that:

​
Where does the key live?

On first boot crewship start auto-generates the key and persists it to <dataDir>/secrets.env — on default installs the same volume as the database, so a copied disk carries both the ciphertext and the key. The server warns about this at startup on every boot, and reports it on the health surface:
To harden, inject ENCRYPTION_KEY from an external secret store and remove the entry from secrets.env.

​
Rotating the master key

crewship admin reencrypt re-encrypts every stored envelope — credential values, refresh tokens, OAuth client secrets, webhook signing secrets, Composio keys, PKCE verifiers, credential escalations — to the current key version, so the old key can be retired instead of living in the environment forever:
The command exits non-zero if any value could not be re-encrypted, so a scripted rotation never retires the old key on a false success. See Credential Encryption for the envelope format, the full column inventory, and backup-bundle caveats.

​
What’s Next

Keeper Security

Set up AI-powered credential access control with security levels L1-L4.

Encryption Details

Deep dive into AES-256-GCM encryption, byte layout, and key versioning.

Orchestration

Create multi-agent missions with credential failover and CooldownManager.

CLI Reference

Complete CLI reference for credential management commands.