Skip to main content

Container Isolation

Crewship uses multiple layers of isolation to prevent agents from accessing unauthorized resources, exfiltrating data, or interfering with other agents.

Isolation Layers

UID Security Boundary

Two fixed UIDs enforce a privilege boundary inside each container:
The UID assignments (1001 for agent, 1002 for sidecar) are a security boundary in the standard runtime image. The exec paths no longer hardcode them, though: when crewshipd runs a command in a container (Keeper /execute, provider Exec), it resolves the container’s actual configured run-as user via ContainerInspect and fails closed — refusing to exec — if that user is empty, undeterminable, or privileged (root uid, root gid, or a root alias). A custom base image with a different non-root uid (e.g. 2000:2000) works automatically; a root-configured container does not get exec’d into.

What Agents Cannot Do

Because agents run as UID 1001 and the sidecar runs as UID 1002:
  • Agents cannot read the sidecar’s in-memory credential store
  • Agents cannot access the IPC token (used for crewshipd authentication)
  • Agents cannot modify network policy configuration
  • Agents cannot tamper with outbound request headers (the sidecar handles this)
  • Agents cannot set their own container ID in Keeper execute requests

The server’s own Docker access

Everything on this page fences the agent out of the host. The Crewship process is on the other side of that fence by necessity: it creates the containers, so it holds Docker API access, which is root-equivalent. You can narrow that to the endpoints Crewship actually calls with a socket proxy — Docker Socket Proxy publishes the exact list and ships it wired in docker/docker-compose.prod.yml. Read its ceiling section before relying on it: the proxy filters URL paths, never request bodies, so it cannot stop Privileged: true. The workspace flag below is what does.

Privileged crews and credentials (fail-closed)

A privileged crew (Docker-in-Docker) runs without the UID 1001/1002 boundary above — under --privileged the split that keeps the sidecar’s CredStore unreadable is gone, so any process in the container can reach it. Because of that, Crewship fails closed (#1032): credentials are not loaded into a privileged crew’s sidecar unless the workspace explicitly opts in via allow_privileged_credentials (default off). The toggle is workspace-scoped and admin-gated (OWNER/ADMIN):
  • UI: Settings → General → Privileged credentials → “Load credentials into privileged crews.” (It is workspace-wide, not per-crew — a crew’s own Settings tab configures that crew, this switch governs the whole workspace.)
  • API: PATCH /api/v1/workspaces/{id} with {"allow_privileged_credentials": true} (read it back on GET /api/v1/workspaces/{id}).
  • CLI: crewship workspace update --allow-privileged-credentials.
Turning this on means you accept that a privileged crew’s agents can read every credential injected into that crew. Leave it off unless you run a DinD workload that genuinely needs vaulted credentials and you trust the code executing inside it.

Container Runtime Security

Crewship supports multiple container runtimes with different security profiles: Configure via:

Container Creation Security

Containers are created with these security measures (internal/provider/docker/docker_container.go):
  • Cap-drop ALL: All Linux capabilities are dropped (CapDrop: ["ALL"])
  • No default cap-add: By default no capabilities are re-added. NET_RAW was removed from the default set (it enables AF_PACKET/DNS-tunneling exfiltration). Devcontainer features may opt into an explicit allowlist (e.g. NET_BIND_SERVICE) via team.CapAdd.
  • SecurityOpt no-new-privileges: Blocks execution of setuid binaries inside the container
  • SUID stripped from the base image: The crewship-sandbox image strips the SUID bit from every setuid-root binary (su, mount, umount, passwd, chsh, newgrp, ssh-keysign, …) at build time. Only sudo is kept — it is the elevation path for the NOPASSWD firewall-init rule and sudo apt-get package-install flows. This is defense-in-depth behind no-new-privileges (which already neuters setuid): it shrinks the surface that a single regression dropping NoNewPrivs or granting a capability could re-arm. A build-time assertion fails the image build if any SUID binary other than /usr/bin/sudo reappears after a base-image bump.
  • Non-root execution: Agent processes run as UID 1001; exec’d commands run as the container’s resolved configured user and are refused outright if that user would be root or cannot be determined
  • Read-only filesystem: Root filesystem is read-only (ReadonlyRootfs: true), relaxed only for privileged crews (DinD)
  • tmpfs /tmp (500M): Writable temp directory (rw,noexec,nosuid,size=500m), not persistent across restarts. rw still permits staging a file here, but noexec blocks executing it directly and nosuid strips any setuid/setgid bit; scripts still run via an explicit interpreter (sh, bash, …), which reads the file rather than exec’ing it.
  • noexec on agent-writable persistent volumes: /workspace, /output and /crew are agent-writable and host-persistent/crew in particular is a host directory that survives container removal and is shared across every agent in the crew. They are mounted noexec,nosuid, so a compromised agent can still write data there but cannot execve() a payload staged on them. This closes the “durable, cross-agent code-execution foothold” vector (a payload dropped once under /crew/shared outliving container rebuilds). Docker’s bind-mount API has no noexec flag, so these ride a bind-backed local volume (type=none,device=<host path>,o=bind,noexec,nosuid) — the mount still binds the exact host directory (data persists, still crew-shared), just with execution denied. The exceptions are deliberate: /opt/crew-tools stays executable because agent tool binaries run from there, and the sidecar/entrypoint binaries are mounted read-only. Scripts under these volumes still run via an explicit interpreter (sh <path>), which reads the file rather than exec’ing it, so the runtime is unaffected. Existing containers created before this policy are recreated on next provision so noexec takes effect. /home/agent is a deliberate exemption — see The /home/agent exemption below.
  • PID limit: 200: Prevents fork bomb attacks (PidsLimit: 200)
  • Memory limits: Configurable per-crew (default_memory_mb, default 8192 MB / 8 GiB)
  • CPU limits: Configurable per-crew (default_cpus, default 2.0)
  • Network isolation: Containers connect to a dedicated Docker bridge network
  • ExtraHosts: host.docker.internal:host-gateway enables container-to-host communication
  • Named volumes: crewship-home-{slug}-{crewID} and crewship-tools-{slug}-{crewID} for persistent home and tools directories (namespaced by crew ID to prevent cross-tenant collisions)

The /home/agent exemption

/home/agent is agent-writable and stays executable. This is a decision, not a gap — the persistence boundary Crewship defends is /crew, not $HOME. The three paths that carry noexec are data paths. /home/agent is the agent’s HOME, and self-installed tooling legitimately runs from it: a pip install --user, a language-server binary, anything on ~/.local/bin. Mounting it noexec would break ordinary agent workflows that /workspace, /output and /crew never supported in the first place. What made /crew worth fencing was a combination /home/agent does not have: A payload staged under /crew/shared was durable and reachable by other agents. One staged in $HOME is neither: it is confined to the crew that wrote it and goes away with crewship admin prune-crew-runtimes or any volume purge. The remaining layers still apply to $HOME: the root filesystem is read-only, all capabilities are dropped, no-new-privileges is set, SUID is stripped from the base image, and egress still passes the sidecar allowlist. Execution from $HOME buys an attacker code execution as UID 1001 — which an agent already has by definition, since running code is what an agent does.
Rely on /crew and /workspace for the durability guarantee, and on the UID boundary plus egress policy for containment. Do not treat $HOME as a no-execution zone: it is not, by design. Tracked and closed in #1462.

Runtime Escape Hatches

A crew’s container can opt out of individual isolation layers through its devcontainer_config. These are the settings with the largest blast radius, so the crew settings UI (Container image & features → Security tab) exposes them as labeled controls with inline warnings instead of a raw JSON blob. Each control serializes to the exact top-level devcontainer_config key the runtime reads, and a raw-JSON “advanced” editor remains available for anything the structured UI does not model (preserved verbatim on save). The UI controls are affordances, not the security boundary — every one is re-validated server-side on save (create and update, crews_create.go / crews_update.go via devcontainer.Config.ValidateSecurity), and the runtime applies the validated values. A caller that bypasses the UI (raw API, a stale client, a hand-edited config) hits the same gate.

How enforcement is wired

The privileged / capAdd / mounts / init keys are top-level devcontainer.json fields — distinct from the capabilities a devcontainer feature declares (those come from arbitrary OCI registries and are force-stripped, internal/devcontainer/features.go). Because they are a first-party operator declaration, the runtime honors them, but only after the save gate has validated them:
  1. Save (ValidateSecurity)privileged requires allow_privileged_credentials (403); capAdd is bounded to NET_BIND_SERVICE (400); every mounts[].source must pass IsAllowedMountSource (400). The keys are persisted in the canonical config (they used to be silently dropped on the auto-inject re-marshal).
  2. Runtime read (ParseConfigSecurity) — at container start the resolver folds the validated top-level keys into the effective requirements the provider maps onto the Docker HostConfig (internal/chatbridge/resolver.gointernal/provider/docker/docker_container.go), re-filtering caps/mounts as defense in depth. The #1032 credential-fail-closed gate reads the same source, so a UI-privileged crew is correctly treated as privileged when deciding whether to load vault credentials.
This means toggling Privileged in the UI actually makes (or, without the workspace flag, refuses to make) the container privileged — the change reaches HostConfig.Privileged, not just the stored JSON. Because the save path is the authority, the capability picker only offers what the save path will accept: NET_BIND_SERVICE is selectable, everything broader renders as privileged only and is inert. A capability stored on a crew from before the gate landed stays interactive so it can be removed, flagged as no longer saveable.

Seeing the posture

A privileged crew is not something you should have to go looking for. The effective posture is surfaced in three places:
  • Crew canvas header — a red Privileged · isolation reduced badge next to the crew name, on the first screen anyone opens.
  • Security tab — an Isolation reduced badge plus a dot on the tab itself.
  • CLIcrewship crew config <crew> --show prints the stored devcontainer_config, privileged included.

Setting them from the CLI

Every control above has a flag on crewship crew config--privileged, --cap-add, --init, --init-hook — so a privileged change is scriptable and reviewable rather than a hand-edited JSON blob. See crewship crew config. The flags merge onto the stored config and hit the identical server-side gate.
A crew running privileged has essentially no container isolation left. The Security tab surfaces an “Isolation reduced” badge on any such crew so the posture is visible wherever the crew is configured. Reserve privileged mode for crews you fully trust (e.g. Docker-in-Docker) and grant a single capability where that is enough.

CLI parity

The same controls are drivable from the CLI, and go through the identical server-side validation:
crewship crew config --init is a deprecated no-op — the PID 1 reaper is always on. The flag still parses, warns, and changes nothing.
These flags merge onto the crew’s stored devcontainer_config (image and features are preserved) and PATCH it back. Uploading a full config file with --devcontainer ./devcontainer.json is validated the same way.

Network Policy

Each crew can have a network access policy:
All outbound connections are allowed. The sidecar still intercepts and injects credentials for known LLM providers, but does not block unknown domains.

Private / LAN endpoints

The SSRF fence blocks RFC1918, loopback and link-local targets in both network modes, so an on-prem Ollama or a self-hosted model endpoint is unreachable by default even for a free crew. Opening it takes two independent opt-ins:
  1. Per crewallow_private_endpoints. Set it from the crew’s Network policy panel (admin-only toggle) or with crewship crew update <crew> --allow-private-endpoints.
  2. Per instance — the operator ceiling CREWSHIP_ALLOW_PRIVATE_ENDPOINTS.
They are ANDed (#974 S5), so a workspace admin cannot self-grant private egress on a shared or cloud host by flipping the crew flag alone — which is exactly the “I set the flag and it’s still blocked” case the UI toggle calls out inline. Link-local and cloud-metadata addresses (169.254.169.254) stay hard-blocked regardless of either setting.
Unknown network modes default to restricted (fail closed). This prevents a typo in the configuration from accidentally allowing unrestricted access.New crews are created restricted by default, and existing crews still on the legacy free default are backfilled to restricted on upgrade (migration v148). A crew that genuinely needs open egress can be re-opened to free, or granted the specific domains it needs.

One allowlist, every egress path

A crew’s network_mode + allowed_domains is a single security dial enforced everywhere the platform makes an outbound request on that crew’s behalf — not only through the agent container’s sidecar proxy. All of these resolve the crew boundary from one shared source (internal/egresspolicy), so a restricted crew cannot exfiltrate through whichever path an author reaches for: For notify and hooks the boundary is keyed on the crew that authored the routine (or owns the hook); the block is applied before any bytes leave — a webhook to a non-allowlisted host is never sent, and an MCP server whose endpoint host is outside the allowlist is never connected. A free crew (the default) is unaffected: every path allows all hosts, exactly as before.
The two in-container rows are guardrails, not containment. Nothing on the server intercepts them — the boundary is the sidecar proxy the process is pointed at, which means the proxy environment is itself the security control. Code that deliberately ignores it (curl --noproxy '*', a raw socket, a non-cooperating binary) still reaches the network, because there is no network-layer fence yet (#1368). Treat these rows as “a well-behaved tool obeys the allowlist”, and the four server-side rows as the ones that hold against hostile code.
Both in-container paths take those variables from one exported source (orchestrator.SidecarProxyEnv), and a routine cannot override them: any *_proxy variable declared in script.env is dropped and logged. The match is on the shape rather than an exact list, because CPython lowercases every environment name before looking for a _proxy suffix — so HtTp_PrOxY would otherwise reach a Python script’s proxy configuration, and .py is a first-class script interpreter.
This table was not always complete. Routine script steps built their exec environment from the step’s own inputs alone and carried no proxy, so a restricted crew reached arbitrary hosts from a script step with a plain curl — found by the in-container red-team suite (scripts/test-harness/test-redteam-insider.sh) and fixed in #1473. The lesson is the one this page keeps making: any new code path that execs into a crew container inherits nothing automatically.
Every non-container path builds its outbound client from the one shared factory, egresspolicy.Client — the pre-flight host check above is a cheap fast-fail, and the client then re-runs the SSRF guard and the crew allowlist on every redirect hop. This is what closes the redirect-exfil bypass (an allowlisted host that 302s to a non-allowlisted one is refused, not followed) and means there is genuinely no per-path divergence: routine http steps, notify/webhook, hooks, and the MCP gateway share the same gated client rather than each hand-rolling a CheckRedirect.

Domain Allowlist Implementation

The DomainAllowlist (internal/egressallow/allowlist.go, a dependency-free leaf so both internal/sidecar and internal/egresspolicy can share it without an import cycle) is a thread-safe set of allowed domain names:
  • Domains are stored lowercase for case-insensitive matching
  • Port numbers are stripped before comparison
  • IPv6 bracket notation is handled correctly
  • *.example.com entries are stored as a subdomain-suffix matcher (matches any subdomain, not the apex); everything else is an exact host match
  • Domains can be added at runtime via allowlist.Add(domain) (exact or wildcard)

Proxy Security

The sidecar proxy (internal/sidecar/proxy.go) enforces several security measures:

Request Body Limits

All proxied requests have their body limited to 10 MB to prevent OOM attacks. LLM API requests are typically under 1 MB.

Hop-by-Hop Header Stripping

Per RFC 2616 Section 13.5.1, the proxy strips these headers: Proxy-Authorization is especially dangerous — an agent could use it to exfiltrate credentials via a controlled proxy.

HTTPS CONNECT Tunnels

For HTTPS requests, the sidecar:
  1. Checks the domain allowlist (in restricted mode)
  2. Establishes a TCP tunnel
  3. Does not inject credentials (the tunnel is opaque)
Credential injection only works for:
  • Plain HTTP proxy requests (agent sets HTTP_PROXY)
  • Reverse proxy mode — Claude Code via ANTHROPIC_BASE_URL=http://127.0.0.1:9119 (→ api.anthropic.com) and, since #1030, Codex via OPENAI_BASE_URL=http://127.0.0.1:9119/openai/v1 (→ api.openai.com) and Gemini via GOOGLE_GEMINI_BASE_URL=http://127.0.0.1:9119/gemini (→ generativelanguage.googleapis.com); the /openai / /gemini routing prefixes are stripped before forwarding. For all three, the real key stays in the sidecar CredStore and only a dummy placeholder reaches the agent env.

Provider Detection

The sidecar automatically detects which LLM provider a request is targeting:

Credential Injection by Provider

Keeper Execute Security

The /keeper/execute flow has the most stringent security because it runs shell commands with credentials:
  1. Sidecar validation:
    • Intent and command length limits (4096 chars)
    • Null byte rejection
    • Shell metacharacter blocking (;, |, `, >, &&, ||, $()
    • Content inside single quotes is exempt
    • Container ID always from IPC config (agent cannot override)
  2. crewshipd validation:
    • Same shell metacharacter checks (defense in depth)
    • Keeper LLM evaluates the command
  3. Execution:
    • Credential injected as environment variable for the command only
    • Output scrubbed of credential values before returning to agent

Output Scrubbing

The scrubber package (internal/scrubber/) removes credential values from agent output. This prevents:
  • Credentials appearing in chat messages
  • Credentials being logged to the progress stream
  • Credentials in Keeper execute output being returned to the agent

Container Auto-Detection

The Docker provider (internal/provider/docker/docker.go) auto-detects the container runtime by probing socket paths:
Each socket gets a 1.5-second ping timeout to avoid blocking on unresponsive daemons.
containerd and nerdctl are not supported. /run/containerd/containerd.sock used to appear in this list. It was removed because containerd serves its own gRPC API over HTTP/2, not the Docker REST API — the client Crewship uses can never talk to it, on any version. If a containerd socket is present when detection fails, the error now names it and says so. Rancher Desktop counts here too: it works in dockerd (moby) mode and not in containerd mode, where it leaves a ~/.rd/docker.sock behind that accepts a connection and then closes it.