> ## Documentation Index
> Fetch the complete documentation index at: https://docs.crewship.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Credentials
> Manage encrypted API keys with priority-based selection, round-robin rotation, and Keeper security gating.
# Credentials
Crewship encrypts all credentials at rest with AES-256-GCM and delivers them to agents through a sidecar proxy -- never as environment variables.
## Credential Types
Canonical list in `internal/api` (`CredentialType` constants). The manifest validator accepts any of these strings on a `credentials[].type:` field; UI and CLI use the same set.
| Type | Description | Example |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| `API_KEY` | LLM provider API key (Anthropic, OpenAI, Google). Auto-injected as the provider's signed header by the sidecar. | `sk-ant-api03-...` |
| `AI_CLI_TOKEN` | OAuth token specifically for an AI CLI adapter (Claude Code, Codex, Gemini CLI). Routed through the sidecar CONNECT tunnel, not the API-key header path. | `sk-ant-oat-...` |
| `CLI_TOKEN` | Non-AI CLI bearer/PAT token (GitHub PAT, npm token, Vercel token, etc.). Delivered as a file under `/secrets/{agent-slug}/` rather than auto-injected on the wire. | `ghp_…` / `npm_…` |
| `OAUTH2` | OAuth 2.0 credential with refresh token. Used for third-party integrations (Google, GitHub, Slack) where the access token expires. | Refresh + access tokens |
| `SECRET` | Generic short secret value (DB password, cookie value, webhook signing key). Encrypted at rest; never auto-injected on the wire. | Database password |
| `GENERIC_SECRET` | Same delivery shape as `SECRET`, distinguished only by the UI so manifest-declared sidecar passwords (`POSTGRES_PASSWORD`, …) are visually grouped apart from app-level secrets. | Sidecar service password |
| `USERPASS` | Username + password pair (HTTP basic, internal services). Stored as a structured value. | `user:secret` |
| `SSH_KEY` | SSH private key in PEM form, mounted into the container at apply-time. | `-----BEGIN OPENSSH PRIVATE KEY-----` |
| `CERTIFICATE` | X.509 client certificate + private key, for mTLS to private APIs. | `cert.pem` + `key.pem` pair |
## Supported Providers
Credentials are associated with a provider for automatic injection:
| Provider | Header Injection | Auto-Detect |
| ----------- | -------------------------------------------------------------------------------- | ----------------------------------- |
| `ANTHROPIC` | `x-api-key` or `Authorization: Bearer` (OAuth tokens starting with `sk-ant-oat`) | `api.anthropic.com` |
| `OPENAI` | `Authorization: Bearer` | `api.openai.com` |
| `GOOGLE` | `?key=` query parameter | `generativelanguage.googleapis.com` |
## How Credential Injection Works
Agents never see raw API keys. The sidecar proxy intercepts outbound HTTP requests and injects credentials based on the destination host:
```
Agent Process (UID 1001)
|
| HTTP request to api.anthropic.com
v
Sidecar Proxy (UID 1002, port 9119)
|
| 1. Match host -> ProviderAnthropic
| 2. Select credential from CredStore
| 3. Inject x-api-key header
v
api.anthropic.com (with real API key)
```
The sidecar also supports a **reverse proxy mode** where agents set `ANTHROPIC_BASE_URL=http://127.0.0.1:9119`. Claude Code sends requests directly to the sidecar over plain HTTP, and the sidecar forwards them to `api.anthropic.com` with the injected key.
## Priority-Based Selection
The `CredStore` (`internal/sidecar/credstore.go`) selects credentials using a two-tier system:
1. **Priority tier:** Credentials with the lowest numeric priority value are selected first (lower = higher priority)
2. **Round-robin within tier:** Multiple credentials at the same priority level rotate to distribute load
```go theme={null}
// CredStore.Select picks the next credential for a provider
// 1. Filter candidates for the requested provider
// 2. Find the best (lowest) priority value
// 3. Round-robin among top-priority credentials
```
The CredStore is an in-memory store -- credentials are never written to disk inside the container. They are loaded at container startup via stdin JSON from the orchestrator.
## Credential Delivery Flow
Credentials are stored in the database encrypted with AES-256-GCM. The format is `v1:base64(IV||AuthTag||Ciphertext)` where IV is 16 bytes and AuthTag is 16 bytes.
When a crew container starts, the orchestrator decrypts assigned credentials using the `ENCRYPTION_KEY` environment variable.
Decrypted credentials are sent to the sidecar process as JSON via stdin -- not environment variables. This prevents credential leakage through `/proc/environ` or `ps aux`.
The sidecar loads credentials into the `CredStore` (a thread-safe in-memory map). The `CredStore` supports concurrent access with `sync.RWMutex`.
When the agent makes an HTTP request, the proxy matches the destination host to a provider and injects the appropriate authentication header.
## Cooldown Management
When a credential receives a `429 Too Many Requests` response, the `CooldownManager` (`internal/orchestrator/`) temporarily removes it from the rotation. This prevents thundering-herd problems when rate limits are hit.
## Credential Security Levels
Credentials can be tagged with security levels for Keeper gating:
| Level | Description | Keeper Behavior |
| ------ | -------------------------------------------- | ------------------------------------ |
| **L1** | Low sensitivity (npm tokens, read-only APIs) | Auto-allow with intent >= 10 chars |
| **L2** | Medium (GitHub write, DB read) | LLM evaluation required |
| **L3** | High (SSH, DB admin, AWS) | LLM evaluation + possible escalation |
| **L4** | Critical (production admin, payment) | Human approval required |
L1 auto-allow **never** applies to `/keeper/execute` requests. Commands must always be evaluated by the Keeper LLM to prevent exfiltration attacks like `echo $TOKEN | base64`.
## Adding Credentials
### Via the UI
Navigate to **Settings -> Credentials** and click **Add Credential**. Enter the name, provider, type, and secret value. The value is encrypted immediately on submission.
### Via Seed Data
For development, set environment variables before seeding:
```bash theme={null}
SEED_ANTHROPIC_API_KEY=sk-ant-api03-... ./dev.sh seed
```
The seeder auto-detects OAuth tokens (prefix `sk-ant-oat`) vs API keys and creates the appropriate credential type.
## Credential Assignment
Credentials are assigned at the workspace level and automatically distributed to agents based on provider matching. Two assignment modes exist:
| Mode | Trigger | Behavior |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Auto-assign** | Agents created from crew templates, via the internal API, or hired as ephemeral agents (`crewship hire` / LEAD-driven spawn) | `autoAssignCredentials` matches provider; best-effort (failures journal `credential.auto_assign_failed`, no Anthropic credential in the workspace journals `credential.auto_assign_empty`) |
| **Manual assign** | Agents created via CLI or UI | User explicitly assigns via `crewship credential assign ` |
## Agent-Proposed Credentials (pending approval)
Agents **cannot activate a credential on their own** — the lifecycle requires a named
human, and the `/secrets/{agent-slug}/` files are read-only (writing one does **not**
register anything in the vault). But an agent that generated a secret for the crew — say
a password for a database it just provisioned — can **propose** it, and a human approves
it with one click.
The agent raises a **`CREDENTIAL` escalation** through the sidecar, carrying the proposed
credential as JSON in the `metadata` field. Send the body over **stdin** (`--data @-`) so the
secret never lands in the shell history or process arguments:
```bash theme={null}
curl -s -X POST http://localhost:9119/escalate \
-H "Content-Type: application/json" --data @- <<'JSON'
{"from":"","reason":"","type":"CREDENTIAL",
"metadata":"{\"name\":\"PG_PASSWORD\",\"type\":\"SECRET\",\"provider\":\"NONE\",\"value\":\"\"}"}
JSON
```
What happens:
1. The value is encrypted and stored **immediately** as a credential with status
`PENDING_APPROVAL`. It shows up in the Credentials page (amber **Pending approval**
badge, under **Needs attention**) and as an item in the [inbox](/guides/inbox), but it
is **never delivered to any agent** while pending.
2. A human **Approves** — from the inbox (one click), the crew escalations panel, or
`crewship escalation resolve --action approve` — and the credential flips to
`ACTIVE`, attributed to the approver. Only now can the crew use it.
3. Or **Rejects** (`--action reject`) and the proposed credential is discarded.
The agent's `/escalate` call blocks until the human approves or rejects (up to 5 minutes).
If the agent does not have the value itself and needs a human to supply it, it omits
`metadata` and describes the need in `context` instead (the legacy human-supplies-the-secret
flow). Provenance is preserved either way: the credential records the **agent** as proposer
and the **human** as approver.
## File-Based Secrets
For non-LLM credentials (database passwords, API tokens), Crewship writes one file per credential to `/secrets/{agent-slug}/`:
```
/secrets/viktor/
.env # Maps: GH_TOKEN=/secrets/viktor/GH_TOKEN
GH_TOKEN # Contains the raw token value
DB_PASSWORD # Another credential
```
The agent reads credentials from these files. The `.env` file maps environment variable names to file paths for tooling compatibility.
## Three Credential Injection Modes
Crewship supports three distinct modes for delivering credentials to agents, depending on the container configuration and credential type.
The standard and most secure mode. Credentials are piped to the sidecar process via stdin as JSON at container startup. The sidecar holds them in an in-memory `CredStore` and intercepts outbound HTTP requests, injecting the appropriate authentication headers based on the destination host.
* Credentials never appear in environment variables or on disk
* The sidecar runs as UID 1002, inaccessible to the agent (UID 1001)
* Supports priority-based selection and round-robin rotation
When the sidecar is disabled, credentials are injected as environment variables using `BuildEnvVars()`. This mode is less secure because credentials are visible in the process environment.
* API keys are set as `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, etc.
* `AI_CLI_TOKEN` credentials with `sk-ant-oat` prefix are mapped to `CLAUDE_CODE_OAUTH_TOKEN`
* Secret-type credentials are excluded from environment variables in sidecar mode
MCP server configurations in `.mcp.json` can reference credentials using `${VAR}` syntax in their `env` blocks. Crewship resolves these references at runtime:
1. The system scans `.mcp.json` for `${VAR}` patterns in env blocks
2. Each `${VAR}` reference is matched against workspace credentials by `env_var_name`
3. Matched credentials are decrypted and injected as actual environment variables
4. Claude Code natively expands `${VAR}` references from the container env vars
```json theme={null}
{
"mcpServers": {
"github": {
"env": {
"GITHUB_TOKEN": "${GITHUB_TOKEN}"
}
}
}
}
```
## OAuth Token Handling
Credentials with the `AI_CLI_TOKEN` type or values starting with `sk-ant-oat` receive special handling:
* The credential is injected as `CLAUDE_CODE_OAUTH_TOKEN` (not `ANTHROPIC_API_KEY`)
* OAuth tokens use an HTTPS CONNECT tunnel through the sidecar proxy, not the API key header injection path
* This distinction ensures that Claude Code authenticates via OAuth flow rather than direct API key authentication
The sidecar detects OAuth tokens by their `sk-ant-oat` prefix and routes them through the CONNECT tunnel handler instead of the standard reverse proxy path.
## Monitoring Credential Exposure
Some credentials unavoidably land in the agent's environment — OAuth tokens
and CLI tokens use a CONNECT tunnel / env-var read that the proxy can't
isolate, and SECRET-level credentials are exposed when Keeper is disabled.
Whenever a credential does land in the agent env, the Orchestrator logs the
exposure so operators can see it and remediate where possible:
* **WARN (actionable)** — a SECRET credential exposed because Keeper is off.
Enable Keeper (`KEEPER_MODEL` / `KEEPER_OLLAMA_URL`) to gate it behind
`/keeper/request` instead.
* **DEBUG (informational)** — OAuth/CLI tokens that are structurally
un-isolatable. Logged for awareness, not remediation.
**Where to find these logs:**
* `crewship agent get ` (debug output / `service_logs`)
* `GET /debug/logs?level=WARN&agent_id=` for programmatic access
## What's Next
Set up AI-powered credential access control with security levels L1-L4.
Deep dive into AES-256-GCM encryption, byte layout, and key versioning.
Create multi-agent missions with credential failover and CooldownManager.
Complete CLI reference for credential management commands.