Skip to main content

​
Internal IPC

Routes mounted under /api/v1/internal/* are not part of the public API. They are the surface used by the sidecar (running as UID 1002 inside every crew container) and a small set of orchestrator helpers. The sidecar holds the only internal-API credential inside a container — a workspace-bound token derived from CREWSHIP_INTERNAL_TOKEN (never the master itself); agents (UID 1001) cannot read it from disk or process memory. This page is here so integrators building their own sidecar image, runtime adapter, or replay tool understand the contract. End users should never call these endpoints directly.
Every route below is wrapped in internalAuth and authenticated with the X-Internal-Token header. Sidecars carry a workspace-bound token (derived per workspace from the master at sidecar start) that only authorizes their own workspace; the token lives only inside the sidecar (UID 1002); agents (UID 1001) cannot read it. These endpoints are not for external clients.

​
Endpoints

One row per documented internal route, grouped by family. See Route catalog for the per-family detail and Authentication for the trust model.
MethodEndpointPurpose
POST/api/v1/internal/cost/recordSidecar reports parsed LLM usage
POST/api/v1/internal/journal/emitSidecar emits a journal entry
GET/api/v1/internal/credentialsList credentials this agent may use
PATCH/api/v1/internal/credentials/{credentialId}Update credential status
POST/api/v1/internal/credentialsSlash-action mirror — create a credential
POST/api/v1/internal/credentials/{credentialId}/rotateSlash-action mirror — rotate a credential
POST/api/v1/internal/chatsCreate a chat row from inside a container
GET/api/v1/internal/chats/{chatId}/resolveResolve a chat ID to its scope
PATCH/api/v1/internal/chats/{chatId}/message-countIncrement message counter
PATCH/api/v1/internal/chats/{chatId}/titleSet chat title
GET/api/v1/internal/agents/{agentId}/resolveResolve agent slug/ID to its full record
GET/api/v1/internal/agents/{agentId}/webhook-secretFetch the agent’s webhook HMAC secret
GET/api/v1/internal/crewsList crews (sidecar discovery)
POST/api/v1/internal/crewsCreate a crew programmatically
POST/api/v1/internal/agentsCreate an agent programmatically
POST/api/v1/internal/agents/hireLEAD-initiated ephemeral hire
GET/api/v1/internal/crew-connectionsList crew connections
POST/api/v1/internal/pipelines/saveSave an agent-authored pipeline definition
POST/api/v1/internal/routines/schedulesSlash-action mirror — create a pipeline schedule
POST/api/v1/internal/skills/generateSlash-action mirror — LLM-author a SKILL.md
POST/api/v1/internal/runsCreate a run row
PATCH/api/v1/internal/runs/{runId}Update run status / metrics
POST/api/v1/internal/crew-messagesSend a peer message
GET/api/v1/internal/crew-messagesList peer messages
GET/api/v1/internal/crew-files/{crewId}Read a file in the crew’s shared volume
POST/api/v1/internal/crew-files/{crewId}Write a file in the crew’s shared volume
POST/api/v1/internal/assignmentsCreate an assignment from inside a container
GET/api/v1/internal/assignments/{assignmentId}Fetch assignment status
POST/api/v1/internal/queriesFree-form sub-query
GET/api/v1/internal/standupFetch the standup digest the agent should read
POST/api/v1/internal/escalationsOpen an escalation
GET/api/v1/internal/escalations/{escalationId}/waitLong-poll for response
POST/api/v1/internal/report-confidenceReport agent self-confidence after a turn
POST/api/v1/internal/missionsCreate a mission
GET/api/v1/internal/missions/{missionId}Get mission details
POST/api/v1/internal/missions/{missionId}/startTransition mission to running
GET/api/v1/internal/issuesList issues
GET/api/v1/internal/issues/{identifier}Get a specific issue
POST/api/v1/internal/issuesCreate issue
PATCH/api/v1/internal/issues/{identifier}Update issue status
POST/api/v1/internal/issues/{identifier}/commentsComment on issue
POST/api/v1/internal/keeper/requestSidecar forwards a credential read request
GET/api/v1/internal/keeper/request/{requestId}Fetch a previously submitted request’s decision
POST/api/v1/internal/keeper/executeSidecar forwards a sealed credential-use request
POST/api/v1/internal/keeper/skill-reviewEvaluate a proposed skill before adoption
POST/api/v1/internal/keeper/behaviorEvaluate an agent behaviour signal
POST/api/v1/internal/keeper/memory-healthEvaluate agent memory health
POST/api/v1/internal/keeper/negative-learningEvaluate a negative-learning candidate
POST/api/v1/internal/mcp-tool-callsRecord an MCP tool invocation for audit
POST/api/v1/internal/port-exposeRequest a capability URL for an internal port

​
Authentication

Every internal route is wrapped in internalAuth: 
X-Internal-Token: <token>
The master token is generated at server startup if CREWSHIP_INTERNAL_TOKEN is not set in the environment. For multi-host deployments, set it explicitly on every host so a sidecar restart on host A doesn’t suddenly fail when its request lands on host B. Sidecars never receive the master token. At sidecar start, the orchestrator derives a workspace-bound token — wsv1.<workspace_id>.<hex(HMAC-SHA256(master, derivation_context || workspace_id))> — and injects it via the sidecar’s stdin IPCConfig. The derivation_context is a fixed, versioned domain-separation string ("crewship internal-token workspace binding v1\x00"); the MAC is computed over that context concatenated with the workspace ID, not over the workspace ID alone. Use internaltoken.DeriveWorkspaceToken (in internal/auth/internaltoken) as the canonical implementation — a custom sidecar that hashes only the workspace ID will mint tokens that fail validation. The internalAuth middleware validates the binding on every request: the MAC is re-derived from the embedded workspace_id and the in-memory master. The binding is enforced as a mandatory request scope:
  • a ?workspace_id query parameter that disagrees with the token’s bound workspace is rejected with 403 before the handler runs; and
  • when the caller omits ?workspace_id, the middleware injects the bound workspace into the query — so every handler that filters by ?workspace_id (/agents/{id}/webhook-secret, /credentials, /agents/{id}/resolve, /chats/{id}/resolve, /crews, …) is tenant-scoped automatically. There is no “legacy unscoped” path for a bound token.
Path-param mutations that don’t read the query (chat message-count / title, run finalize, credential status) additionally constrain their lookup by the bound workspace; a foreign-tenant row returns 404, never mutated. Handlers scoped by a workspace_id carried only in the JSON body (/cost/record, /journal/emit, /pipelines/save, and the issue / mission / assignment / query / escalation create handlers and the confidence report) enforce the binding in-handler via assertInternalTokenWorkspace (403 on a foreign tenant), since the auth middleware cannot inspect bodies. Derivation is stateless, so tokens stay valid for the lifetime of one server boot and roll automatically with the master on restart. The unbound master token remains valid for host-side trusted callers (chat bridge, webhook secret resolver, LLM proxy monitor) that never enter a container — but only from a loopback origin: a master token arriving from a Docker-bridge / LAN IP is refused with 403 (capping the blast radius of a master copy leaked into a container). Set CREWSHIP_INTERNAL_ALLOW_ANY=true to relax both the loopback pin and the network gate when a reverse proxy rewrites RemoteAddr. Workspace / crew / agent / mission scope is not taken from request bodies in a way that can cross tenants. The IPC layer reads it from the sidecar’s IPCConfig (set by the orchestrator at exec time and pinned to the container) and projects it onto the request, and the token binding constrains every handler to its workspace. An agent that captured the sidecar’s token cannot forge cross-tenant attribution: the token only authorizes the workspace it is cryptographically bound to.
The /api/v1/internal/keeper/* family (skill-review, behavior, memory-health, negative-learning) reads body.workspace_id AND ctx.workspace_id (the latter set by the internalWsCtx middleware from the ?workspace_id query parameter). Historically this family carried a documented cross-tenant gap. Both halves are now closed:
  • Asymmetric forgery (workspace A in query, workspace B in body) is rejected at the handler boundary by the assertBodyWorkspaceMatchesCtx helper in internal/api/keeper_phase2.go.
  • Symmetric forgery (caller picks one foreign workspace consistently across query + body) is closed by PR-F24: the X-Internal-Token a sidecar holds is bound to its workspace, and both internalAuth and internalWsCtx reject a ?workspace_id that disagrees with the binding.
The remaining cleanup, PR-F25, drops body.workspace_id from these four handlers entirely and derives workspace from context only — an architectural simplification, no longer a security gap.

​
Route catalog

Sourced from the per-domain internal/api/router_*.go files (split by domain in May 2026 — see router_internal.go for the internal IPC surface). Methods and paths are exact.

​
Cost ledger

MethodPathPurpose
POST/api/v1/internal/cost/recordSidecar reports parsed LLM usage. See Cost record below.

​
Journal

MethodPathPurpose
POST/api/v1/internal/journal/emitSidecar emits a journal entry. Same structure as the in-process journal.Entry.

​
Credentials

MethodPathPurpose
GET/api/v1/internal/credentialsList credentials this agent is allowed to use.
PATCH/api/v1/internal/credentials/{credentialId}Update credential status (e.g. mark expired).
POST/api/v1/internal/credentialsSlash-action mirror — create a credential. Requires X-Caller-User-Id. See Slash-action mirrors.
POST/api/v1/internal/credentials/{credentialId}/rotateSlash-action mirror — rotate a credential. Requires X-Caller-User-Id. See Slash-action mirrors.
The POST create/rotate routes are registered only when the public CredentialHandler is wired (router_internal.go:80-84); test routers that omit it skip the mirror.

​
Chats

MethodPathPurpose
POST/api/v1/internal/chatsCreate a chat row from inside a container (e.g. webhook-triggered).
GET/api/v1/internal/chats/{chatId}/resolveResolve a chat ID to its scope (workspace, agent, etc.).
PATCH/api/v1/internal/chats/{chatId}/message-countIncrement message counter.
PATCH/api/v1/internal/chats/{chatId}/titleSet chat title (used by webhook-created chats).

​
Agents and crews

MethodPathPurpose
GET/api/v1/internal/agents/{agentId}/resolveResolve agent slug/ID to its full record.
GET/api/v1/internal/agents/{agentId}/webhook-secretFetch the agent’s webhook HMAC secret (used by the webhook handler to verify signatures).
GET/api/v1/internal/crewsList crews (sidecar discovery).
POST/api/v1/internal/crewsCreate a crew programmatically.
POST/api/v1/internal/agentsCreate an agent programmatically.
POST/api/v1/internal/agents/hireLEAD-initiated ephemeral hire — sidecar /spawn proxies here. See Ephemeral hire.
GET/api/v1/internal/crew-connectionsList crew connections.
The POST .../agents/hire route is registered only when the public AgentHandler is wired (router_internal.go:60-63); the nil-safe adapter returns 500 otherwise.

​
Pipelines, routines, and skills

These are the trusted forward-targets for agent-authored pipelines and the sidecar’s slash-action routes. Each dispatches into the same public handler the dashboard / CLI uses, with workspace + role context injected at the internal boundary.
MethodPathPurpose
POST/api/v1/internal/pipelines/saveSave an agent-authored pipeline definition. See Pipeline save.
POST/api/v1/internal/routines/schedulesSlash-action mirror — create a pipeline schedule (routine). See Slash-action mirrors.
POST/api/v1/internal/skills/generateSlash-action mirror — LLM-author a SKILL.md. See Slash-action mirrors.
routines/schedules is registered only when the PipelineHandler is wired (router_internal.go:72-75); skills/generate only when the SkillGenerateHandler is wired (router_internal.go:76-79).

​
Runs

MethodPathPurpose
POST/api/v1/internal/runsCreate a run row. Idempotent on (workspace_id, trace_id).
PATCH/api/v1/internal/runs/{runId}Update run status / metrics.
Since PR #234, runs are reconstructed from journal entries; these endpoints exist for transitional callers and emit a run.* journal entry as a side effect. New integrations should emit journal entries directly via /api/v1/internal/journal/emit instead.

​
Crew messaging and files

MethodPathPurpose
POST/api/v1/internal/crew-messagesSend a peer message.
GET/api/v1/internal/crew-messagesList peer messages.
GET/api/v1/internal/crew-files/{crewId}Read a file in the crew’s shared volume.
POST/api/v1/internal/crew-files/{crewId}Write a file in the crew’s shared volume.

​
Assignments and queries

MethodPathPurpose
POST/api/v1/internal/assignmentsCreate an assignment from inside a container.
GET/api/v1/internal/assignments/{assignmentId}Fetch assignment status.
POST/api/v1/internal/queriesFree-form sub-query (e.g. agent asking another agent).
GET/api/v1/internal/standupFetch the standup digest the agent should read.
POST/api/v1/internal/escalationsOpen an escalation.
GET/api/v1/internal/escalations/{escalationId}/waitLong-poll for response.
POST/api/v1/internal/report-confidenceReport agent self-confidence after a turn.

​
Missions and issues

MethodPathPurpose
POST/api/v1/internal/missionsCreate a mission.
GET/api/v1/internal/missions/{missionId}Get mission details.
POST/api/v1/internal/missions/{missionId}/startTransition mission to running.
GET/api/v1/internal/issuesList issues.
GET/api/v1/internal/issues/{identifier}Get a specific issue.
POST/api/v1/internal/issuesCreate issue. author_agent_id (validated against the crew/workspace) is persisted as the creator with authored_via = agent_tool_call; optional author_chat_id / author_run_id record provenance.
PATCH/api/v1/internal/issues/{identifier}Update issue status. Writes a mission_activity audit row attributed to agent_id (or system when absent). A comment requires agent_id.
POST/api/v1/internal/issues/{identifier}/commentsComment on issue. Requires agent_id — unattributed comments are rejected with 400.

​
Keeper

MethodPathPurpose
POST/api/v1/internal/keeper/requestSidecar forwards a credential read request.
GET/api/v1/internal/keeper/request/{requestId}Fetch the status / decision of a previously submitted request.
POST/api/v1/internal/keeper/executeSidecar forwards a sealed credential-use request after gatekeeper approval. See the Keeper section for the full request flow.
Keeper Phase 2 (aux-LLM evaluators, PR-C / PRD §6 F4). All four are wrapped in internalAuth and internalWsCtx (which reads ?workspace_id= into the request context — see the Authentication known-exception note). They are always registered, returning a deterministic 503 (“evaluator not configured”) until the evaluators are wired (router_internal.go:164-183).
MethodPathPurpose
POST/api/v1/internal/keeper/skill-reviewEvaluate a proposed skill before adoption.
POST/api/v1/internal/keeper/behaviorEvaluate an agent behaviour signal.
POST/api/v1/internal/keeper/memory-healthEvaluate agent memory health.
POST/api/v1/internal/keeper/negative-learningEvaluate a negative-learning candidate.

​
MCP audit

MethodPathPurpose
POST/api/v1/internal/mcp-tool-callsRecord an MCP tool invocation for audit.

​
Port expose

MethodPathPurpose
POST/api/v1/internal/port-exposeSidecar requests a capability URL for an internal port.
User-facing port-expose lifecycle endpoints live on the public API — see Port Expose.

​
Endpoint deep-dives

The routes above are catalog entries; the sections below give the full request/response contract for the ones with non-trivial bodies, trust models, or error matrices.

​
Cost record

POST /api/v1/internal/cost/record
The sidecar’s write target after parsing an LLM response. Validates the request, calls paymaster.Record (which inserts the cost_ledger row and emits llm.call + optionally cost.incurred), then paymaster.EnforceQuota (which emits budget.warning / budget.exceeded based on the parsed rate-limit headers). Request body: body cap 16 KiB. 
{
  "workspace_id": "ws_…",
  "crew_id": "crw_…",
  "agent_id": "agt_…",
  "mission_id": "MIS-42",
  "provider": "anthropic",
  "model": "claude-opus-4-7",
  "input_tokens": 12483,
  "output_tokens": 4521,
  "cached_input_tokens": 1024,
  "cache_creation_tokens": 0,
  "billing_mode": "metered",
  "subscription_plan": "",
  "quota_remaining_pct": 0.42,
  "quota_window": "tokens",
  "had_status_429": false
}
FieldTypeNotes
workspace_idstringRequired. The sidecar fills this from its IPCConfig (set by crewshipd at container boot), so an agent that captures the token still cannot forge a row for a foreign workspace.
crew_id, agent_id, mission_idstringOptional scope. Same trust model — sidecar fills from IPCConfig.
provider, modelstringRequired. Used for rate-card lookup.
input_tokens, output_tokensintegerOptional, default 0. Providers occasionally omit usage blocks; we record the audit row anyway rather than dropping. Negative values clamped to 0 by Estimate.
cached_input_tokens, cache_creation_tokensintegerOptional, default 0.
billing_modestringmetered (default), flat_rate. Anything else → 400.
subscription_planstringRequired when billing_mode=flat_rate. Display label like "Anthropic Max".
quota_remaining_pctnumber0–1. Smallest of parsed quota windows.
quota_windowstringDisplay label: requests, tokens, input-tokens, … A non-empty value is the sentinel “rate-limit headers were present”, which gates EnforceQuota.
had_status_429boolWhen true, triggers budget.exceeded regardless of quota_remaining_pct.
Fields the sidecar has no authority over are derived server-side:
Server-derivedHow
Ledger IDGenerated in paymaster.Record.
TimestampServer now (UTC).
cost_usdpaymaster.Estimate(provider, model, ...) for metered; forced to 0 for flat_rate.
cost_confidenceprecise if input_tokens > 0 || output_tokens > 0 (heuristic — the sidecar saw a usage block), else estimate. Forced to unknown for flat_rate by paymaster.Record.
tagsAlways {"source": "sidecar"}. Caller cannot override.
Response: 202 Accepted 
{
  "id": "cl_a1b2c3d4"
}
The handler writes the ledger row synchronously, then runs paymaster.EnforceQuota (best-effort — its result does not affect the response code, only the journal). The response body is intentionally minimal; the journal entries (llm.call, cost.incurred, optionally budget.warning / budget.exceeded) are where the operator-visible artifacts land. When had_status_429=true, the response is still 202 — the ledger row was written and budget.exceeded was emitted. The caller (sidecar) is responsible for propagating the upstream’s 429 back to the agent; this endpoint does not echo the 429 because it succeeded at recording the cost. Errors:
StatusCondition
400Invalid JSON, body exceeds 16 KiB, missing workspace_id / provider / model, billing_mode not metered / flat_rate, or any validation wrapped as paymaster.ErrInvalidRequest (e.g. flat_rate without subscription_plan).
401Bad or missing X-Internal-Token.
500DB error from paymaster.Record.

​
Ephemeral hire

POST /api/v1/internal/agents/hire
The internal entry for a LEAD agent spawning a short-lived “contractor” agent. The sidecar’s /spawn flow proxies here. The HireInternalAdapter reads workspace_id from the query string (the sidecar attaches it), injects a MANAGER role into the request context, then calls the public AgentHandler.Hire path unchanged — so the per-crew autonomy policy gate, the crews.max_ephemeral_agents quota, the audit log, and the inbox emission all run exactly as they do for a human hire. The adapter deliberately does not inject a user_id; the resulting audit row is attributed to actor.system. (internal/api/internal_hire.go:45) Request body (same shape as the public POST /api/v1/agents/hire):
FieldTypeNotes
crew_id / crew_slugstringOne required, mutually exclusive.
template_slugstringRequired. Template the ephemeral is cloned from.
modelstringOptional. Empty falls back to the template default.
ttl_minutesintegerOptional. Clamped to 1..1440; 0 → 30 min default.
reasonstringRequired (audit + history trail).
parent_lead_idstringOptional. Must be a LEAD in the same crew.
workspace_id is not read from the body — it comes from the ?workspace_id= query parameter the sidecar attaches. Response: mirrors the public Hire handler — 201 Created (live ephemeral), 202 Accepted (waiting on inbox approval), with a body carrying id, slug, status, ephemeral, expires_at, decision, and optionally inbox_item_id. Errors: 400 (missing workspace_id query param, or missing/invalid body fields), 403 (strict crew rejected the hire), 404 (crew or template not found), 429 (per-crew ephemeral quota reached), 500 (adapter not configured / DB error).

​
Pipeline save

POST /api/v1/internal/pipelines/save
The trusted endpoint the sidecar forwards to when an agent emits a new pipeline definition. X-Internal-Token runs upstream; the handler trusts the caller’s claim about author identity from the body. It parses + validates the DSL, runs cross-crew reference checks and cycle detection over the workspace’s saved pipelines, then persists. (internal/api/pipelines_crud.go:634) Request body:
FieldTypeNotes
workspace_idstringRequired.
slugstringRequired.
definitionJSONRequired. Raw pipeline DSL.
name, descriptionstringOptional metadata.
author_crew_id, author_agent_id, author_chat_id, author_run_idstringAuthor attribution; recorded with authored_via = agent. author_crew_id also scopes the agent-slug validation set.
last_test_run_atstringRFC 3339. Feeds the test-run gate.
last_test_run_passedboolFeeds the test-run gate.
Response: 201 Created — the saved pipeline object (toPipelineResponse). Errors: 400 (invalid JSON, or missing workspace_id / slug / definition), 409 (slug already exists in the workspace), 422 (DSL parse / validation / cycle-detection failure, or the test-run gate was not satisfied), 500 (DB error).

​
Slash-action mirrors

Four internal routes mirror public handlers so the sidecar’s slash-action surface (and, where noted, autonomous agent tool calls) have a trusted backend to proxy into. Each adapter reads workspace_id from the ?workspace_id= query parameter and injects a role into the request context before dispatching to the shared public handler. They follow a dual-path model (PRD-SLASH-CAPABILITIES-2026 §6.5):
  • User-initiated (X-Caller-User-Id header present): the adapter gates on the caller’s per-action capability (routine.create, skill.create, credential.create, credential.rotate) and stamps the real user id for audit attribution.
  • Autonomous-agent (X-Caller-User-Id absent): falls through the capability gate (the autonomy gate runs upstream), relying on the injected role to clear the public handler’s pre-existing role check.
RoutePublic handlerInjected roleX-Caller-User-IdCapability
POST /api/v1/internal/routines/schedulesPipelineHandler.CreateScheduleMANAGEROptionalroutine.create (when present)
POST /api/v1/internal/skills/generateSkillGenerateHandler.GenerateMANAGEROptionalskill.create (when present)
POST /api/v1/internal/credentialsCredentialHandler.CreateADMINRequiredcredential.create
POST /api/v1/internal/credentials/{credentialId}/rotateCredentialHandler.RotateADMINRequiredcredential.rotate
Credentials are the exception: both credential mirrors reject with 401 when X-Caller-User-Id is absent — autonomous-agent credential mutation is intentionally not supported, because the public Create/Rotate handlers write a human user id into the audit / rotation-initiator columns and rotation has a workspace-wide blast radius. (internal/api/internal_credentials_mutate.go:99-113) Request / response shapes match the underlying public endpoints:
  • routines/schedules → body is the schedule shape (name, target_pipeline_slug or target_pipeline_id, cron_expr, timezone, inputs, enabled); cron_expr is required. Returns 201 Created with the schedule object. (internal/api/internal_routines.go:76, internal/api/pipeline_schedules.go:82)
  • skills/generate → body { "slug", "prompt", "model"? } (slug + prompt required). The adapter stamps workspace_id as the {workspaceId} path value the public handler expects. Returns 200 OK with { "skill_id", "slug", "content", "scan_status", ... }. Needs an Anthropic API_KEY credential in the workspace (412 otherwise). (internal/api/internal_skills.go:49, internal/api/skills_generate.go:94)
  • credentials (create) → same body as Create Credential. Returns 201 Created. (internal/api/internal_credentials_mutate.go:49)
  • credentials/{credentialId}/rotate → same body as Rotate Credential (value required, grace_seconds optional). credentialId is read from the path. Returns 200 OK with the rotation object. (internal/api/internal_credentials_mutate.go:76)
Common errors across all four: 400 (missing workspace_id query param or body validation), 401 (credentials only: missing X-Caller-User-Id), 403 (capability denied), 500 (adapter not configured).