Skip to main content

​
Troubleshooting

Most beta-era problems fall into a small set of buckets. Start here before opening an issue.

​
First step: crewship doctor

doctor runs a battery of checks and prints one row per check with a status: PASS / WARN / FAIL / INFO. Read each WARN/FAIL — the hint text in the second column is the remediation. doctor covers (as of v0.1 beta):
  • Container runtime detected — Docker / Podman / Colima / OrbStack / Apple Containers. If FAIL, install one (see Install — container runtime requirement).
  • Server reachable — TCP-dials the configured server host:port (default localhost:8080) so you can tell “server down / wrong port” apart from “auth issue”.
  • Database file permissions — confirms ~/.crewship/crewship.db is 0600 (owner-only). Drift here usually means a backup was restored via tar without preserving the mode bit.
  • Telemetry status — shows current opt-in state and resolved endpoint host (vendor default vs CREWSHIP_SENTRY_DSN override).
  • DSN reachability — best-effort TCP connect to the Sentry endpoint:443 (5s timeout). Sentry being unreachable is not a Crewship health signal — WARN here just informs you that crashes won’t ship until the network heals.
  • Update available — checks GitHub Releases API for a newer version. WARN if a stable release is newer than your installed build.

​
Which container runtime is Crewship actually using?

Admin → Runtime lists every container runtime present on the host — each with its own product mark, its own daemon’s version, and its own socket — and marks the one this server is driving In use. Everything else is badged Detected: installed, not driving anything. The panel is read by an admin (the endpoint redacts socket paths and daemon versions below ADMIN), and refreshes when you press Re-detect. It does not poll: each check re-probes every candidate socket, and the answer only changes when someone installs a runtime. Two distinctions the panel makes that nothing else can:
  • The product, not the socket. OrbStack, Rancher Desktop and Docker Desktop all offer to point /var/run/docker.sock at their own daemon. Detection follows the link, so a machine running OrbStack says OrbStack, not Docker.
  • Present vs in use. If runtimes are listed but none says In use, this server started without a container provider — --no-docker, or one that failed to start. Agents cannot run until it has one, and that is a different problem from having no runtime installed.
There is no runtime switcher, in the console or anywhere else. container.provider accepts docker, apple or auto — there is no value for OrbStack, Colima, Rancher or Podman, because all of them are reached through the same Docker API and Crewship uses whichever socket answers first.To change which one it uses, there are exactly two levers:
Apple Containers is the one real choice, because it is a separate provider rather than another Docker-API daemon:
The same inventory is available over HTTP, and to any tool that can read JSON:
runtime / version / socket summarise the entry with in_use set, and are null when runtimes are installed but none is in use. One daemon produces one entry even when it answers on several paths. install_links lists every runtime Crewship can drive, so the runtimes you have not installed are still named; containerd and nerdctl are deliberately absent, because containerd serves a gRPC API that Crewship’s Docker client cannot speak.

​
Symptom catalog

Find your symptom below; each panel expands to its diagnosis and fix.
This is an in-band failure: the agent CLI exited cleanly and then said, in its own final stream event, that the turn failed — a refusal, an internal CLI error, or an exhausted quota. Crewship marks those runs error instead of completed, because the alternative is a mission or routine continuing on an empty answer while the run looks green and still gets billed.The error message quotes the CLI’s own text. To confirm and find the cause:
  1. Open the run in the journal and find the exec.command end entry. An in-band failure has severity warn with exit_code: 0 and in_band_error: true — that pair is the fingerprint.
  2. Read the result event’s metadata on the same run; it carries the CLI’s terminal envelope with usage and cost.
In chat, the turn is still persisted: the agent’s text (a refusal, or whatever it managed to say) plus an error part carrying the reason, so a reload shows the failure rather than an empty bubble. A delegated assignment keeps the sub-agent’s output in result_summary for the same reason. In a routine, the step fails and — under the default on_fail: escalate_tier — walks to the next fallback tier. It is not retried on the same tier: an in-band failure is deterministic, so retrying it just bills for the same answer (details).Common causes by message:
  • turn cap (N turns) — the agent ran out of agent-loop turns mid-task. Raise max_turns on the step, or split the work. Note that the step still walks the fallback-tier chain, so a task that flails for 20 turns can replay them on a pricier model; on_fail: abort stops that.
  • a refusal (“I cannot help with…”) — reword the task, or lower the tool profile’s blast radius.
  • a usage / quota message — check Paymaster and the credential’s plan.
  • an internal CLI error — the vendor CLI in the crew image may be stale; reprovision the crew.
Per-adapter signal shapes and the run-level vs tool-level boundary are documented in CLI Adapters → In-band failures. A failed tool call does not fail a run.
The interactive path opens a WebSocket to the server. When the connection is refused after the upgrade (the 101 succeeds, then the socket closes), the CLI now prints the server’s reason instead of a bare ws read: EOF — for example:
Match the reason to the fix:
  • invalid or expired ws-token — the short-lived WS ticket (15 min) expired or didn’t validate. Just retry; the CLI mints a fresh ticket per run. Persisting means clock skew or a JWT-secret mismatch between the CLI’s server target and the server.
  • no auth message received … (handshake timed out) — the upgrade succeeded but the client’s auth frame never arrived in time. Usually a proxy buffering the first WS frame, or a stale CLI. Check any reverse proxy in front of /ws streams frames without buffering, and run crewship self-update.
  • session_revoked — your browser session was revoked; re-authenticate.
On the server, every post-upgrade rejection now logs a WARN line — grep the server log for ws connection rejected after upgrade to see the reason and remote_addr. If you also see a version-skew warning when the CLI fetches its ws-token, upgrade the stale side first.
This is the LLM provider connection dropping mid-generation — a network blip or the provider load-shedding — not a bug in your prompt or agent. Before this check existed, a cleanly-closed connection with no terminal event from the provider (no message_stop for Anthropic, no finish_reason for OpenAI-compatible backends, no NDJSON line with "done":true for Ollama) looked identical to a normal completion, and the run silently succeeded with truncated content: half-written code, cut-off JSON, no error anywhere. For Ollama it was worse than mislabeled success — the aggregated response content came back completely empty even though tokens had streamed in, because it was only assembled once the terminal line arrived. All three now surface as a real failure instead.The error message always ends in unexpected EOF, which the pipeline’s transient-error matcher recognizes — the same task tier is retried automatically the way a 500/502 from the provider would be. You only need to act if it keeps failing after retries: check the provider’s status page, or (for a custom openai_compat endpoint, or a self-hosted Ollama behind a proxy) confirm the proxy isn’t buffering/timing out chunked responses before the model finishes.This is unrelated to a deliberate cancellation (stopping a run, or a timeout) — those still surface as a cancellation, never as this message.
Almost always means migrations failed silently or the data directory isn’t writable. Run:
--verbose switches the logger to debug level. Look for failed to run migrations or failed to open database. Common fixes:
  • Permission error: chmod 700 ~/.crewship && chmod 600 ~/.crewship/crewship.db. The migrate path enforces 0600 on the DB file but can’t reset 0700 on the parent.
  • Disk full: ~/.crewship/crewship.db lives next to a pre-migration snapshot (*.pre-migrate-*.bak). If the disk is full, the snapshot can’t be written and the boot aborts to prevent a half-migrated DB. Free space, then re-run.
crewship start (and crewship doctor) distinguish two situations and print the matching fix for your OS:
You already have a runtime — it just isn’t running. The preflight names what it found and how to start it, e.g. open -a Docker (Docker Desktop), colima start, container system start (Apple Containers), or sudo systemctl start docker on Linux.
Nothing is installed. Quick install paths on macOS:
Colima, Rancher Desktop, Docker Desktop and podman machine all run the daemon inside a VM that only sees the host directories it was configured to share — Colima shares $HOME by default and nothing else. Crewship keeps every host path it hands the daemon under its data dir (~/.crewship by default, inside that share) precisely so a stock install works without touching the VM’s mount list. If you move the data dir with --data-dir / CREWSHIP_DATA_DIR, put it somewhere the VM shares — see “A bind mount fails for a file that exists” below.On Linux, curl -fsSL https://get.docker.com | sh, then sudo systemctl enable --now docker and add yourself to the docker group (sudo usermod -aG docker $USER, log out and back in). Or install podman. On Windows, install Docker Desktop with the WSL 2 backend. Crewship picks whatever socket it finds — no further config needed.
The file is on your disk and the daemon still says it is missing. That is not a missing file — it is a VM share-set problem. Colima, Rancher Desktop, Docker Desktop and podman machine run the daemon inside a VM, and a bind source is resolved inside that VM. A path the VM does not share does not exist as far as the daemon is concerned.Crewship names this for you: the error says which runtime, which path, and the command that shares it — e.g.
The same check runs at startup, so a runtime that cannot see the mandatory binds says so in the server log before any crew is woken.Two things worth knowing:
  • crewship-sidecar and entrypoint.sh are staged automatically. They are resolved next to the crewship binary (/opt/homebrew/... for Homebrew, /usr/local/bin for install.sh) — outside every default VM share set. Crewship copies them into <data-dir>/output/.runtime/ at startup and binds the copies, so the only host subtree the daemon has to see is the one it already needed for /workspace, /output and /crew. Nothing to configure.
  • The remaining case is a data dir outside the share set. If you point --data-dir / CREWSHIP_DATA_DIR at, say, /tmp on macOS (which is /private/tmp, and is not the VM’s own /tmp) or a path on an external volume, add that directory to the VM’s mounts or move the data dir back under $HOME.
Starting Colima, launching Rancher Desktop or repointing DOCKER_HOST moves Crewship to a different daemon — it does not stop what is running on the old one. Those containers keep their credentials and stay bind-mounted to the same host /crew, /workspace and /output directories the new containers use, so an agent inside one can go on writing the crew memory the live crew reads back.At startup Crewship now enumerates every other Docker-API daemon reachable on the host, finds crew containers belonging to this instance (matched on the instance’s container prefix), and stops them, logging what it stopped and where:
They are stopped, never removed — the container, its logs and its volumes stay for a post-mortem. To keep one running while you investigate:
which reports the same information and prints the docker stop command instead of running it. The container keeps write access to live crew memory until you do.
When the runtime can’t start, the chat now streams a classified error with a machine-readable code in the event metadata, instead of a generic “failed to start agent container”. The full raw cause is in the run journal / server logs.
The server compares the configured CREWSHIP_SIDECAR_PATH binary against its own build at boot. This WARN means the sidecar is older than the running server, so it was not rebuilt/recopied for this deploy — crew containers are bind-mounting a stale sidecar, and sidecar-side features shipped since (egress client, the memory-auth chokepoint, token_fp used by orphan-container reaping) may be running from an older binary.Fix — rebuild and recopy the sidecar to the path the server actually uses, then restart agents so new containers pick it up:
Restarting agents without recopying the binary remounts the same stale file, so the rebuild step is the one that matters. On dev slots dev.sh rebuilds the sidecar alongside the server on every start and forces CREWSHIP_SIDECAR_PATH to that fresh, co-located binary (ignoring any stale pin in .env.local, since #1402), so this warning should not normally appear there — if it does, the co-located build step itself was skipped or failed; re-run the deploy and check the build output.
Specific to Claude Code adapter. Almost always missing or invalid CLAUDE_CODE_OAUTH_TOKEN — the agent CLI failed its startup auth check. Fix:
Other non-zero exits (1, 2, 127) usually mean the agent CLI binary isn’t installed in the container image — check crewship agent debug <id> for the container’s resolved PATH.
Wrong shell. Use bash, not sh:
The script uses set -euo pipefail and array-expansion guards that Debian/Ubuntu dash (the default /bin/sh) does not implement. See the bash note in Install.
Update brew and try again:
If the tap repo (https://github.com/crewship-ai/homebrew-tap) lists a formula version different from what you’d expect, the goreleaser auto-push for the latest release may have failed — file an issue and fall back to the curl | bash install.
Releases are cosign-signed but not Apple-notarized in v0.1 (Apple Developer Program enrollment is a separate workstream). After the first download you’ll see:
Unblock once:
Or right-click → Open → Open Anyway in Finder. The brew install path strips quarantine automatically; only direct downloads need this.
crewship start as a systemctl --user service (or a tmux-less SSH session) is killed when the user logs out, because systemd-user sessions are tied to the login by default.Enable lingering so the user manager keeps running:
loginctl show-user "$USER" | grep Linger= reports Linger=yes once it’s on. Headless servers (no interactive shell) are the common case; desktop installs that always have a logged-in graphical session can skip this step.
Windows builds aren’t Authenticode-signed in v0.1 (we sign later in the beta). On first launch you’ll see a SmartScreen “Don’t run” prompt.Right-click crewship.exe → Properties → check Unblock → OK. One-time per binary — subsequent runs skip the prompt. The Homebrew and Docker Compose install paths bypass this entirely.
This means your local DB has a migration applied under one name but the binary expects a different name at that version. Caused by upgrading from a fork or a pre-release build that shipped a renamed migration.
Try the repair first — it is not destructive. When the two names are both migrations this binary knows about, the number recording one of them is simply wrong, and moving it fixes the boot without touching a single table:
The repair refuses if the database names a migration this binary does not have at all. That means a different Crewship migrated it, not a renumbered one, and no renumbering can make this binary’s assumptions true. Then, and only then:
Restore your last backup with the matching binary (crewship db restore-snapshot), or install a binary new enough to include the migration. Wiping the DB is the last resort and destroys your data — move it aside, never delete it.
Maintainer-side: two guards cover this now. The Migration Lint CI workflow catches a colliding version pre-merge, and new migrations take timestamp version numbers (YYYYMMDDHHMMSS), so two branches can no longer reach for the same one. See Migrations.
Token expired or never set. Two paths:
In the UI: log out and log back in. If the bootstrap user was reset (rare — only happens after crewship.db deletion), visit /bootstrap to recreate the initial admin account.
When a lead delegates work ([Assignment] @agent is working on the task… in the lead’s chat), the sub-agent run lives in the server process. If the server crashes or restarts mid-run, that run is gone — and before v0.1.x the assignment row stayed RUNNING forever, which also blocked one of the crew’s concurrency slots so every later delegation for that crew queued indefinitely.This is now self-healing, on two layers:
  • On restart, the server fails every assignment that was RUNNING under the previous process with the reason interrupted by server restart. The lead’s chat resolves with the failure, the slot is freed, and any queued assignments behind it are dispatched immediately.
  • While running, a background sweeper (every 5 minutes) fails any assignment stuck in RUNNING with no completion past its staleness bound: the target agent’s configured timeout_seconds plus a 15-minute grace margin, with a 2-hour floor. Because the bound tracks each agent’s own timeout, a long-but-healthy run is never touched — even one configured to run longer than the floor; the sweeper only reclaims slots leaked by abnormal conditions (e.g. a force-killed container that never reported back).
If you still see a permanently spinning assignment on an older build, restarting the server after upgrading clears it. Both transitions land in the Crew Journal as assignment.failed entries attributed to assignment_recovery, so a post-mortem can tell recovery-failures apart from real agent failures.
WARN, not FAIL. Telemetry won’t ship until the network heals; nothing else breaks. If you want to silence the warning:
Or redirect to a Sentry instance you can reach:
If a routine is created under a slug that previously existed and was deleted (routine delete is a soft-delete), older builds resurrected the tombstoned database row — and with it the previous routine’s invocation count, last-run status, and journal entries. A brand-new routine could appear in the UI with “10 invocations, last FAILED” before it had ever run, typically right after crewship seed --nuke or when an agent re-authors a routine you deleted.Current builds purge the stale counters and journal entries when a deleted slug is reused, so the recreated routine starts clean. On older builds the inherited history is cosmetic — it corrects itself after the routine’s next real run.

​
Where to go next

If crewship doctor is green and you still hit issues:

Open a GitHub issue

Attach crewship doctor --no-color output and the relevant log section.

Discord community

Real-time help, especially for “is this expected?” questions.

Crash-report dashboard

If telemetry is enabled and the crash has a Sentry event ID, paste it into the issue — the maintainer can correlate immediately.

Source-map view

For frontend issues, the Sentry stack trace has a “View on GitHub” link per frame — useful screenshots to attach to reports.