Consolidate
Two background workers run against the Crew Journal. Both start automatically at server boot (consolidate.StartBackground) on the cadences below; you can also trigger a consolidation run on demand via the API/CLI.
- Consolidator — runs every 6h. Walks recent journal entries per crew and produces a set of learned rules appended to
.memory/topics/learned-YYYY-MM-DD.md. Emitsmemory.consolidated. - Compactor — runs daily at 03:00 UTC. Rolls up low-signal entries older than 30 days into a single
system.compactionentry and deletes the originals.warn/errorrows are preserved indefinitely.
The “supervisor crew” orchestration pattern (a dedicated crew that reviews and curates consolidated rules) is on the roadmap and not part of the current automatic workers.
Candidate entry types
The consolidator considers these entry types for rule extraction:Rules output
Rules land in{memoryRoot}/{crewSlug}/topics/learned-YYYY-MM-DD.md. Format:
Summarizer
SummarizerClient is an interface:
Which model runs it
The summarizer resolves thecurator auxiliary slot — the slot the Judge
models card and crewship keeper aux list have always described as “Skill
review + memory consolidation”. Repointing it applies to the next
consolidation, not the next restart:
anthropic default on an instance with no ANTHROPIC_API_KEY — consolidation
falls back to the boot-time KEEPER_OLLAMA_URL + KEEPER_MODEL client, so an
install running a local judge with no API key keeps consolidating. Only when
there is neither is the summarizer absent.
The curator slot’s --timeout bounds the slot’s evaluator (skill review)
and the operator-model sweep; it is deliberately not applied to a consolidation
prompt, which is batch work over up to a few hundred journal entries and is
bounded by the provider’s own client timeout instead.
If no summarizer is configured at all:
- Background runner skips every crew silently (the pin-snapshot path still runs).
- Manual trigger returns 202 Accepted with
note: "no summarizer configured, skipping"and emitssystem.consolidation_triggered+system.consolidation_completed(status=skipped-no-summarizer) so the audit trail captures the no-op.
Compactor
Compaction target:info and notice entries older than 30 days that are NOT type summary.generated, memory.consolidated, or any checkpoint/fork/approval/regression. Rolls up into one system.compaction entry per (crew, day) bucket with a count and kind breakdown in the payload, then archives the originals to journal_entries_archived (with the payload truncated to 400 chars) before deleting from the base table. Done in a single transaction so neither side can drift.
Preserved indefinitely (never compacted):
warnanderrorseveritysummary.generated,memory.consolidated- Everything under
approval.*,checkpoint.*,fork.*,eval.regression_detected - Anything tagged with
priority='permanent'(see Crew Journal — Priority markers)
Archive layer (PR #212)
Migration 55 addedjournal_entries_archived so compaction is no longer destructive. The aged row is copied into the archive with its summary truncated to 200 chars and its payload truncated to 400 chars (stored in the compressed_payload column); the pre-truncation size is recorded in original_size_bytes. There is no trailing marker — the copy is a plain substr. The archive table has the same workspace/ts indexes as the base, so historical forensic queries still work — they just have to include the archive table:
/journal view does not read from the archive by default — operators have to explicitly opt in via ?include_archive=true. This keeps the live UI fast and the archive as a forensic-only surface.
If you want to tune journal compaction retention, edit RunnerOptions.CompactionOlderThan and restart. Per-workspace retention for memory_versions (a separate Iter 4 worker sweep) is configurable at runtime via PATCH /api/v1/admin/memory/config — see Memory Observability — Per-workspace retention.
Manual trigger
OWNER or ADMIN only. Returns 202 Accepted immediately; the actual run happens in a 10-minute background goroutine.
Response shapes:
since accepts Go durations (90m, 24h) plus shorthand d (days) and w (weeks). Default 24h if unset or unparseable. Soft-deleted crews are treated as not found — a deleted crew can’t grow fresh memory artifacts via explicit ID.
CLI
crewship consolidate.
Journal entries emitted
Gotchas
- Ollama must be reachable when the
curatorslot points at it (and for the episodic indexer, which always does). SetOLLAMA_MODELS="/Volumes/SSD 990 PRO/ollama-models"andollama servebefore starting the server locally. Consolidation itself is no longer Ollama-only: acuratorslot on a hosted provider works with no Ollama at all. - The triggered journal entry is always emitted. Even the “no summarizer” skip path emits both triggered + completed so operators see the full round-trip. If you filter for
memory.consolidatedonly, you’ll miss skipped runs. - Per-workspace in-flight guard. A second manual trigger for the same workspace while the first is running returns 409. Track
worker_idif you need to wait for completion. - Compactor is aggressive on info/notice. If you rely on
exec.commandentries older than 30 days for forensics, export them out-of-band before the Compactor rolls them up.
Related
- Crew Journal — input stream.
- Episodic memory — the vector index built from the entry types these workers emit.
- Memory System — consumes the generated
topics/learned-*.mdfiles. crewship consolidate, Consolidate API.