Skip to main content

kind: Routine

What it is

kind: Routine declares a workspace-scoped declarative AI workflow — what Crewship has historically called a “pipeline” in the database and a “routine” in the UI. A routine is a versioned, schedulable, webhook-dispatchable DAG of steps (agent runs, HTTP calls, code blocks, transforms, waits) that the workspace’s agents can invoke and that humans can trigger directly via cron or webhook. This kind subsumes the legacy crewship routine save -f X.json flow while remaining fully backward-compatible: the JSON body that the old CLI sent under --file is exactly the shape that lives under spec: (modulo the manifest-only schedules + webhook fields described below). An operator migrating from JSON to YAML can copy their existing routine.json body verbatim into spec:, add the manifest envelope (apiVersion, kind, metadata), and have it apply identically. No field renames, no semantic drift — the internal/pipeline.Parse server-side validator is the same code path in both paths. Routine is the biggest kind in the manifest system because one document atomically deploys three sibling rows:
  1. one pipelines row (the routine definition itself)
  2. zero or more pipeline_schedules rows (cron triggers)
  3. zero or one pipeline_webhooks row (public dispatch token)
A single crewship apply -f routine.yaml creates / updates / prunes all three in one transaction-like sequence — what used to take three CLI calls (routine save + routine schedule create + routine webhook create) is now one declarative file.

YAML schema

Examples

Minimal — single-step routine, no triggers

Realistic — Discord hourly sync with cron + webhook

CLI reference

REST endpoint mapping

How each manifest field lands on a REST call and ultimately a DB column: Plan-then-Apply order is fixed: routine row first, then schedules, then webhook. Schedules and the webhook foreign-key back to the routine, so we cannot reverse the order.

Validation rules

Validate (client-side, before any REST call fires):
  • metadata.name and metadata.slug required.
  • metadata.labels.crew required; must appear in the workspace’s declared crews OR remote crews.
  • spec.dsl_version required.
  • spec.steps must contain at least one step.
  • Every step.agent_slug (on type: agent_run steps) must appear in the workspace’s declared agents OR remote agents. We DO NOT validate the agent-membership-in-parent-crew constraint here; the server enforces that at apply time.
  • Every schedule must have a non-empty name; names unique within the document.
  • Every schedule’s cron must parse with github.com/robfig/cron/v3’s standard parser (5 fields + descriptor support).
  • Every schedule’s timezone must parse via time.LoadLocation.
  • Webhook token_env_ref is NOT validated here — workspace creds aren’t in WorkspaceContext today. A Plan-time advisory line on the report flags missing resolution; Validate stays purely structural so Export → re-apply round-trips cleanly.
The DSL itself (step shapes, validation blocks, outcomes, etc.) is re-validated server-side by internal/pipeline.Parse and internal/pipeline.Validate. We deliberately don’t re-implement that logic here — duplicating it would just create skew opportunities.

Apply behavior

ApplyUpsert (default)

  1. Routine row. Look up /pipelines/{slug}:
    • missing → Action=Create, POST /pipelines/save
    • drifted (name, description, or canonical definition JSON differs) → Action=Update, POST /pipelines/save again (the save endpoint is idempotent on slug; it bumps the version)
    • identical → Action=Unchanged
  2. Schedules. List /pipeline-schedules filtered by slug, match-by-name against the declared schedules. For each:
    • declared, not on remote → Action=Create, POST /pipeline-schedules
    • declared and drifted (cron, timezone, enabled, or inputs differ) → Action=Update, PATCH /pipeline-schedules/{id}
    • declared and identical → Action=Unchanged
    • on remote but no longer declared → Action=Delete, DELETE /pipeline-schedules/{id} (the manifest is the source of truth)
  3. Webhook. GET the routine’s webhook (if any):
    • declared enabled: true, no remote → Action=Create, POST /pipeline-webhooks
    • declared, remote drifted → Action=Update, which is delete-then-recreate (no PATCH endpoint exists)
    • declared and identical → Action=Unchanged
    • webhook: omitted (or enabled: false) but remote exists → Action=Delete

ApplyStrict

Refuses to update or delete; if any routine in the manifest already exists on the server, apply stops with an error. Use in CI when “this manifest must create fresh resources” is the requirement.

ApplyReplace

Destructive recreate: emits Action=Delete for every existing routine + schedule + webhook that matches a manifest slug, then creates everything fresh. The webhook token + signing secret change on replace (server-minted on each create) — operators must re-distribute the new public URL after a replace.

Round-trip via export

crewship export crew <crew-slug> emits one Routine document per routine where pipelines.author_crew_id resolves to <crew-slug>. Each document includes the full DSL under spec: plus every nested schedule and the (optional) webhook block. What round-trips losslessly:
  • metadata.name, metadata.slug, metadata.labels.crew
  • The entire routine DSL (spec.dsl_version, spec.description, spec.inputs, spec.steps, spec.credentials_required, spec.estimated_cost_usd, spec.estimated_duration_seconds, spec.max_cost_usd, spec.egress_targets)
  • spec.schedules[] (every field — name, cron, timezone, enabled, inputs)
  • spec.webhook.enabled only
What does NOT round-trip (manifest-only fields):
  • spec.webhook.require_token — the server stores signing_secret_set instead; export emits require_token: true implicitly via the default.
  • spec.webhook.token_env_ref — purely a Plan-time hint to the CLI about which env var holds the public token; not persisted.
Operators editing an exported document should re-add token_env_ref manually before re-applying, otherwise the Plan layer will print a warning that the webhook’s public URL won’t be discoverable from the env.

Code steps: wired runtimes (expr, cel) vs shell runtimes (rejected)

type: code supports two wired, deterministic, token-zero runtimes. Shell runtimes (bash | python | go) are rejected at author time because no sandbox runner is wired.

runtime: expr — single comparison, token-zero

The expr runtime (internal/pipeline/runner_code_expr.go) is a pure-Go, in-process evaluator: no container, no LLM, no filesystem or network — it honours the token-zero guarantee and adds no code-execution surface. It evaluates a single comparison and emits true / false:
Operands are numeric or string literals, or a CREWSHIP_INPUT_<NAME> env reference. This is the wake-gate / cost-spike primitive — pair it with a schedule whose wake_gate checks the probe output.

runtime: cel — general agentless logic, token-zero

The cel runtime (internal/pipeline/runner_code_cel.go) evaluates a Google CEL expression. CEL is non-Turing-complete (every expression provably terminates), pure-Go, and sandboxed by construction — no loops, no I/O — so it keeps the token-zero / no-RCE guarantees while giving you real logic: boolean operators (&&, ||, !), arithmetic, string ops, list/map membership, ternaries, and field access. It is the primitive to reach for when expr’s single comparison is not enough. Inputs are exposed as the typed inputs map variable (numbers stay numbers), so reference them directly — no {{ }} needed:
A bool result emits true / false; numeric and string results emit their canonical string form. Compile/eval errors (unknown variable, bad syntax) fail closed.

runtime: bash | python | go — rejected at author time

These runtimes are schema-legal names but have no sandboxed runner wired in this build. As of PR #710 a routine that uses one is rejected when you save / apply / test_run it — it can no longer save-cleanly-then-fail-at-3am:
crewship apply also surfaces the same gap as a plan-time warning for any routine that bypassed the validator (legacy import bundles, direct API writes). If you need real shell, use the conversion recipe below.

Conversion recipe (shell runtimes → agent_run)

Replace the code step with an agent_run against an agent whose tool_profile: FULL (or any profile that includes shell). The agent runs the same command from inside its container, which is already wired end-to-end. Before:
After:
Trade-off: an agent_run is ~30× more expensive than a raw shell exec because it goes through the LLM. For pure shell probes that’s acceptable as a stopgap — but for a real, multi-file program (a PDF parser, a reconciliation script) the proper primitive is a type: script step (below), which runs your bundled script directly, token-zero.

Script steps (type: script) — bundled scripts, deterministic, token-zero

A script step runs a bundled script that already lives in the crew’s shared dir (/crew/shared) directly inside the crew container — the same hardened sandbox (non-root 1001, --cap-drop=ALL, no-new-privileges, read-only rootfs) the crew’s agents run in. Unlike code steps (inline source, restricted to the expr/cel runtimes) a script step points at a real file, so it’s the right shape for a multi-file program with dependencies. It is deterministic and token-zero — no LLM in the loop — the first-class replacement for “an agent_run whose prompt tells the model to shell out to a script.”
  • Path resolves under /crew/shared.. traversal and absolute paths outside the shared dir are rejected at author time.
  • Interpreter is inferred from the extension (.pypython3, .shbash, .jsnode, .rbruby, .gogo run, …) unless you set it explicitly.
  • Inputs flow two ways: args and env values are template-substituted ({{ inputs.x }} / {{ steps.y.output }}) exactly like an agent_run prompt, and every declared routine input also arrives as CREWSHIP_INPUT_<NAME>. interpreter, path, and args are assembled into an argv (never a shell string), so arguments cannot inject.
  • Output: the script’s stdout becomes the step output (flows to {{ steps.parse.output }}) — write only your payload (e.g. strict JSON) to stdout and diagnostics to stderr; a non-zero exit code fails the step (stderr is surfaced in the error). Every script exec is recorded as an exec.command journal entry (command + exit code + duration) for audit.
Delivering the script. The script must exist in /crew/shared before the run. The declarative way is the crew manifest’s files: block — the script travels WITH the workspace and re-applies on every crewship apply:
(See docs/manifest/crew.md “spec.files[]”. Max 1 MiB per file.) For a one-off push to a live crew, crewship crew files save <crew> shared/scripts/parse_vypis.py --file scripts/parse_vypis.py still works — but it does not survive the next rebuild the way the manifest block does. Caveat — egress. Per-step egress is not enforced at exec: a script inherits the crew container’s global network policy (only http steps honor egress_targets). Keep network-touching logic in http steps, or scope the crew’s allowed_domains.

Approval gates (type: wait, kind approval)

A wait step with kind: approval pauses the run for a human decision. The run does NOT block the caller: when a foreground routine run reaches the gate it returns promptly with status WAITING and a waitpoint token, and the run row is parked (status=waiting) — it has released its execution slot.
Resolve it from the inbox UI or the CLI:
Approving (or rejecting) resumes the run from the gate: completed steps are restored and skipped, the wait step resolves from the recorded decision, and the rest of the routine runs. A parked run also survives a server restart — the boot-time resume scan re-enters waiting runs. (Timeouts on a parked approval are reconciled at the next boot scan rather than live.)

See also

  • Your First Crew — the parent concept; routines reference a crew via metadata.labels.crew. step.agent_slug resolves against agents-in-crew.
  • Connectorcredentials_required types are resolved against workspace credentials, which may come from installed connectors.
  • Hookpre_run / post_run hooks fire around every routine invocation.
  • schemas/routine.v1.json — JSON Schema for the DSL portion of spec: (everything except schedules + webhook). Use with VSCode / JetBrains autocomplete: add "$schema": "./schemas/routine.v1.json" to a standalone routine.json file.

Run observability: tags, metadata, replay, errors

Runs carry trigger.dev-style observability surface for filtering and post-failure recovery.

Tags + metadata at invoke

  • Tags — workspace-scoped labels (max 10/run, lowercased). Surfaced on the run detail; group related runs (incl. replays, which inherit the source run’s tags).
  • Metadata — a JSON object stored on the run and returned by GET /pipeline-runs/{id}. Set at invoke today; mid-run mutation + {{ run.metadata.X }} templating is a follow-up.

Replay a failed run

Re-invokes the routine with the run’s captured inputs. The new run is stamped is_replay=true + replay_of=<run_id>; a step can skip side effects on replay by gating on {{ env.is_replay }}:

Errors view + bulk replay

Failed runs are bucketed by a stable error fingerprint (failing step
  • normalized message), so like failures group together:

Waitpoint callback tokens (external completion)

A wait step parks the run on a high-entropy token. Beyond the inbox approve/reject flow, an external system can complete the wait via a public callback URL — no workspace JWT, the token is the auth (same model as webhook dispatch). Surface the URL from the pending waitpoint:
The external task then completes (or denies) the wait:
approved defaults to true (bare POST = “task done, continue”). payload is stored on the waitpoint for the resumed step. Endpoint: POST /api/v1/waitpoint-tokens/{token}.

Batch trigger

Fan out N runs of one routine from an array of input sets. Every run is tagged batch:<id> so the set is retrievable.
inputs.jsonl is one inputs object per line (or a JSON array). Endpoint: POST /pipelines/{slug}/run_batch (max 50 items/batch). The run-level --tag / --metadata apply to every run in the batch.

Per-step prompt/model override (no version bump)

Tweak a single step’s prompt or model without bumping the routine version — the override is applied at run start over the versioned DSL. The durable, versioned routine stays the source of truth; the override is a thin live patch an operator can set and clear.
Only non-empty fields win, so a prompt-only override leaves the authored model. Endpoints: PUT|DELETE /pipelines/{slug}/steps/{stepId}/override, GET /pipelines/{slug}/overrides.

Deferred dispatch: delay, ttl, debounce, priority

A trigger that carries a delay or a debounce key is parked in pending_runs instead of running immediately; an in-process dispatcher (5s tick) fires due rows highest-priority-first and expires rows past their ttl. Immediate runs (no delay/debounce) are unchanged. The user who enqueued the deferred trigger is carried onto the fired run, so a notify step’s to: trigger reaches them when it finally runs — not a workspace-wide fallback.
  • --delay N — fire N seconds out (returns SCHEDULED with a pending id).
  • --ttl N — expire the deferred run if not dispatched within N seconds.
  • --debounce-key K — repeat triggers sharing K extend the window + replace inputs (one run fires); --debounce-window (default 30) sets the window, --debounce-max caps total extension so a hot key still fires.
  • --priority N — higher fires first among due deferred runs.
API: POST /pipelines/{slug}/run accepts delay_seconds, ttl_seconds, debounce_key, debounce_window_seconds, debounce_max_seconds, priority, idempotency_key_ttl_seconds. GET /pipelines/pending, POST /pipelines/pending/{id}/cancel.
Note: priority orders the deferred dispatch queue. Immediate runs execute on arrival, so priority there is recorded but not consumed until the per-crew admission queue (QUEUE-MECHANISM) lands.

Lifecycle hooks (before_all / after_all / on_failure)

Routine-level hooks run deterministic side-channel steps around the main execution — a clean home for setup/teardown that isn’t a pipeline step. Hook steps must be code | http | transform (no agent_run: a hook must not recurse or spend tokens).
Semantics: before_all runs first; if it fails the run is marked FAILED and the steps never execute. after_all runs after a COMPLETED run, on_failure after a FAILED run — both best-effort (logged, never change the run’s outcome). Hooks fire only on the top-level run (not nested call_pipeline expansions) and are skipped on resume re-entry + dry-run. http hooks live inside the same security perimeter as http steps: the routine’s egress_targets and the authoring crew’s network policy gate the host (redirects included), and credential_ref.type resolves against the workspace credential vault by type at run time.

Declarative cron

Cron triggers are already declarative + versioned with the routine via spec.schedules (cron expr + IANA timezone, see “YAML schema” above) — a crewship apply -f routine.yaml deploys the steps + schedules + webhook atomically, so a schedule change is version-controlled alongside the logic. The in-process scheduler (30s tick) fires them.