Migrations Catalog
Crewship runs Go-only migrations against SQLite. The full ordered list lives ininternal/database/migrate.go (var migrations), and they apply in version order at startup, tracked in the _migrations table for idempotency.
This page is a flat reference for the recent set — useful when reading server logs, restoring from a backup that pre-dates a feature, or auditing what shipped between two releases. For older migrations (v1–v49) read migrate.go directly; this catalog focuses on the Crew Journal era and onward.
v50–v53 — Backup & Crew Journal foundations
| v | Name | Adds | PR |
|---|---|---|---|
| 50 | add_instance_config | Per-instance config row (replaces env-only configuration). | — |
| 51 | add_port_exposures | port_exposures table — capability URLs for in-container ports. | — |
| 52 | add_crew_journal | 8 tables: journal_entries, journal_embeddings, agent_status, approvals_queue, checkpoints, hooks_config, cost_ledger, budget_limits. | #204 |
| 53 | add_eval_runs | eval_runs (Quartermaster’s durable replay/regression index with status + metrics columns). | #204 |
journal_entries is the canonical audit stream. After v53, replay/regression analytics have a durable home.
v54–v55 — Memory uplift
| v | Name | Adds | PR |
|---|---|---|---|
| 54 | add_memory_importance_and_gate_rewards | journal_embeddings.importance_score, reference_count, last_referenced_at. Foundations for nightly decay-and-reinforce. | #211 |
| 55 | add_memory_quality_uplift | journal_entries_fts (FTS5 mirror), journal_entries_archived (compaction sink with truncated payloads), memory_relations (embedding relation graph), memory_health_snapshots (daily 5-metric scores). | #212 |
?q= parameter on the journal API is backed by the FTS5 table from v55.
v56–v59 — Chat UI overhaul
| v | Name | Adds | PR |
|---|---|---|---|
| 56 | add_journal_ws_crew_ts_index | Composite (workspace_id, crew_id, ts) index for the Timeline tab’s crew-filtered queries. | #225 |
| 57 | add_chat_extras | Four tables: chat_branches (edit-and-resend branch tree), message_reactions (per-(chat, message, emoji, user) aggregated to counts), chat_attachments (per-chat link row with sha256 + storage path), workspace_files (durable workspace-scoped blob index). | #225 |
| 58 | add_user_preferences | user_preferences (user_id, pref_key, pref_value) — generic per-user JSON-blob KV. First consumer is bottom-panel height in /crews; the schema is intentionally generic so further UI settings land without another migration. | #225 |
| 59 | add_chats_origin | chats.origin (UI | CLI | WEBHOOK | CRON | AGENT, whitelisted in the handler — no DB CHECK) for grouping in the Sessions sidebar. NULL = pre-migration / unrecognised value; UI omits the chip rather than guessing. | #225 |
v60–v61 — Unified journal / Runs SSOT
| v | Name | Adds | PR |
|---|---|---|---|
| 60 | add_journal_ws_trace_index | Composite partial index (workspace_id, trace_id) WHERE trace_id IS NOT NULL. Makes GROUP BY trace_id (run aggregation) cheap. | #234 |
| 61 | drop_agent_runs | Idempotent backfill from agent_runs → journal_entries, snapshot to agent_runs_archive, then DROP TABLE agent_runs. | #234 |
journal_entries is the single source of truth for runs. journal.ListRuns reconstructs the row shape via aggregation; the legacy table is gone but its content survived in agent_runs_archive for forensic reads.
The five new entry types (run.started, run.completed, run.failed, run.cancelled, run.timeout) shipped with the same PR.
v62 — Paymaster billing modes
| v | Name | Adds | PR |
|---|---|---|---|
| 62 | add_paymaster_billing_modes | 9 columns on cost_ledger: billing_mode, quota_remaining_pct, quota_window, subscription_plan, four rate_*_per_m rate-card snapshot columns, cost_confidence. Plus partial index (workspace_id, billing_mode). | #232 |
PR #232 originally numbered this migration v60. The merge with PR #234 (which had taken v60 + v61 for the unified journal) renumbered it to v62 — see the
Renumbered from v60 to v62 comment in migrate.go. Documentation that pre-dates the merge may still mention v60; the canonical number is v62.cost_ledger distinguishes metered API-key calls from flat-rate subscription calls and snapshots the rate card per row. See Paymaster.
v63–v99 — gap (read migrate.go directly)
Migrations between v63 and v99 covered routine maintenance, manifest layer expansion, credential vault iterations, eval-runs widening, and the CLI token tier refresh. Read
internal/database/migrate.go directly — these are documented inline at the constant/comment level and don’t compose into a single feature story worth a catalog entry. The Agent Evolution stack picks up at v100 below.v100–v107 — Agent Evolution stack (PR-B → PR-G/PR-F)
The Agent Evolution PRD (§6 inPRD-AGENT-EVOLUTION-2026.md, internal spec) ships across 8 migrations. They are functionally orthogonal — operators upgrading from a pre-v100 build to v107 land all eight at once and the system end-state is correct; operators upgrading through intermediate versions also work because each migration is additive (no destructive renames after v104).
| v | Name | Adds | PR |
|---|---|---|---|
| 100 | rbac_extensions | Extends the RBAC tables with the manage_governance capability bit used by PR-B’s policy + PR-G’s self-learning gates. Plus per-user audit triple (set_by_user_id / set_at / reason) consumed by every subsequent governance migration. | — |
| 101 | autonomy | crews.autonomy_level (TEXT enum: strict|guided|trusted|full, default guided) + crews.behavior_mode (TEXT: warn|block, default warn) + audit triple. CHECK rejects the forbidden combination (autonomy=full + behavior_mode=block) at the DB layer; UI mirrors. | #461 (PR-B) |
| 102 | keeper_phase2 | Widens keeper_requests.request_type CHECK to admit four new kinds (skill_review, behavior, memory_health, negative_learning). Adds skill_invocations audit table + skills.lifecycle_state enum (active|stale|archived|deprecated). | #470 (PR-C) |
| 103 | ephemeral_agents | Five additive columns on agents: ephemeral (BOOL), expires_at, expired_at, parent_lead_id, hire_reason. Plus crews.max_ephemeral_agents (default 10) + a partial index (expires_at) for sweeper performance. | #469 (PR-D) |
| 104 | persona_rename | The one destructive rename: agents.system_prompt → agents.system_prompt_legacy. Companion code in internal/memory/persona.go::BackfillFromLegacy drains the legacy value into PERSONA.md on first write per agent. | #471 (PR-E) |
| 105 | peer_consent | GDPR primitives for per-user peer cards: user_peer_consent (opt-out), peer_cards (disk-mirror index), peer_card_audit (per-action audit keyed by data subject). | #471 (PR-E) |
| 106 | self_learning | agents.self_learning_enabled (BOOL, default 0) + audit triple. Consumed by F4.4 negative-learning + F6 persona-suggest ALLOW paths — OFF queues a blocking inbox row, ON auto-applies. | #472 (PR-G) |
| 107 | gdpr_cascade | data_subject_id foreign key on memory_versions + inbox_items (both nullable, partial indexes). New gdpr_actions audit table (Article 15 + Article 17 history per subject). CHECK requires non-empty length(trim(reason)) when action='delete'. | #472 (PR-G/PR-F) |
| 112 | user_models | Disk-mirror index for the evolving per-operator model: user_models keyed on UNIQUE(workspace_id, user_slug) — no agent_id, because the model is per (operator, workspace), not per (agent, operator). Nullable crew_id (ON DELETE SET NULL) records which crew’s shared memory holds the file. user_id cascades. Reuses user_peer_consent for opt-out and peer_card_audit for write/delete records. | (PR #10) |
Migration ordering notes — the v100–v107 renumber cascade
Migration ordering notes — the v100–v107 renumber cascade
v100–v107 landed across multiple PRs in parallel and several were renumbered during merge to avoid version collisions:
- PR-D
ephemeral_agentswas originally drafted as v100 on its branch. Bumped to v102, then to v103 as PR-B (v101 autonomy) + PR-C (v102 keeper_phase2) merged ahead. Final landed number onmainis v103. - PR-E
persona_rename+peer_consentwere originally v102 + v103 on their branch. Renumbered to v104 + v105 after PR-C and PR-D merged. - PR-G
self_learning+gdpr_cascadewere drafted as v106 + v107 and kept those numbers — by the time they merged, the renumber cascade was already settled.
Why user_models is v112, not v111
Why user_models is v112, not v111
The migration runner keys on the version number, not the name — so two
branches that both claim the same next number will land non-deterministically
depending on merge order, and the second one silently skips.
user_models was
developed alongside a conversation-search branch that takes the next free
number (v111); to dodge the collision, user_models deliberately claims
max+2 = v112. The gap at v111 is intentional and harmless: the runner only
requires versions to be monotonic, not contiguous.Self-learning gate behaviour (v106)
Self-learning gate behaviour (v106)
A common confusion: v106 only RECORDS the per-agent
self_learning_enabled flag. The downstream consumers (internal/api/keeper_phase2.go::HandleNegativeLearning, internal/api/agent_persona.go::SuggestAgentPersona) read the flag at decision time, not at migration time. Flipping the flag mid-flight via the PATCH /api/v1/agents/{id}/learning endpoint takes effect on the NEXT evaluator run — no agent restart, no migration re-apply.The audit triple (self_learning_set_by_user_id / self_learning_set_at / self_learning_reason) is non-NULL on every flip. The API handler enforces non-empty reason; the column itself is NULLable so the v106 backfill (default 0 for existing rows) doesn’t have to invent a fake actor.GDPR cascade backfill gap (v107)
GDPR cascade backfill gap (v107)
v107 adds the
data_subject_id columns but does NOT backfill them for pre-v107 rows. Existing memory_versions and inbox_items rows from before the upgrade stay NULL and miss the GDPR cascade. Tracked as PR-F32 — see PRD §6.1 for the deferred-but-tracked status. An operator upgrading from an older Crewship to a v107+ build should run the cascade against new SAR requests with full confidence, and treat historical data as “needs manual review” until PR-F32 lands.The gdpr_actions audit table is the canonical record. See GDPR — Article 15 + Article 17 for the operator playbook.How to read this in operation
Whencrewshipd starts, it logs each migration as it applies:
The
_migrations table records the applied set and refuses to apply a version-name pair that disagrees with what the code expects — this catches the classic “two PRs both claimed version N with different SQL” footgun loudly at startup.Restoring from a backup that pre-dates a column
The backup subsystem (internal/backup/runner.go) records the migrations applied at backup time. On restore, every migration the target has but the source lacked runs its restoreBackfill hook (when defined) against the freshly-inserted rows — so a restore from a v59 bundle into a v62 server populates the new columns sanely instead of leaving them at the SQL DEFAULT. Pure ADD COLUMN migrations that rely on the DB default need no hook; complex ones (e.g. backfilling a JSON column) provide one.
Related
- Architecture — Schema footprint.
- Crew Journal — what v52, v55, v60, v61 mean for the audit stream.
- Paymaster — what v62 means for cost accounting.
- Episodic memory — what v54 + v55 mean for recall.
- Chat & Sessions — what v57–v59 mean for the chat surface.