​

Episodic Memory

​

Embedding criteria

// internal/episodic/types.go
func shouldEmbed(entryType string, severity string) bool {
    switch entryType {
    case "peer.escalation",
         "summary.generated",
         "memory.consolidated",
         "approval.denied",
         "eval.regression_detected":
        return true
    case "keeper.decision":
        return severity == "warn" || severity == "error"
    case "mission.status_change":
        return severity == "warn" || severity == "error"
    }
    return false
}

​

Storage

CREATE TABLE journal_embeddings (
  entry_id           TEXT PRIMARY KEY REFERENCES journal_entries(id) ON DELETE CASCADE,
  vector             BLOB NOT NULL,     -- raw float32 bytes
  dim                INTEGER NOT NULL,
  model              TEXT NOT NULL,     -- "nomic-embed-text", etc.
  indexed_at         TEXT NOT NULL,
  importance_score   REAL DEFAULT 0.5,  -- fed into recall sort (see "Importance scoring")
  reference_count    INTEGER DEFAULT 0, -- bumped when a hit lands in a prompt
  last_referenced_at TEXT               -- stamped at the same time
);

​

Migration 55 additions

-- FTS5 contentless mirror of journal_entries.summary + payload.
-- Insert/update/delete triggers keep it in lockstep with the base table.
-- Powers the ?q= parameter on /api/v1/journal and the BM25 leg of HybridRecall.
CREATE VIRTUAL TABLE journal_entries_fts USING fts5(
  summary, payload,
  content='journal_entries', content_rowid='rowid',
  tokenize='porter ascii'
);

-- Compaction sink. Aged rows from journal_entries land here with the payload
-- copied into compressed_payload (truncated to 400 chars) before being
-- deleted from the base table; original_size_bytes records the pre-truncation size.
CREATE TABLE journal_entries_archived (
  id                  TEXT PRIMARY KEY,
  workspace_id        TEXT NOT NULL,
  crew_id             TEXT,
  agent_id            TEXT,
  mission_id          TEXT,
  ts                  TEXT NOT NULL,
  archived_at         TEXT NOT NULL,
  entry_type          TEXT NOT NULL,
  severity            TEXT NOT NULL,
  priority            TEXT NOT NULL DEFAULT 'normal',
  actor_type          TEXT NOT NULL,
  actor_id            TEXT,
  summary             TEXT NOT NULL,
  compressed_payload  TEXT NOT NULL DEFAULT '{}',  -- payload truncated to 400 chars
  original_size_bytes INTEGER NOT NULL DEFAULT 0
);

-- embedding relation graph: similarity + curated edges linking related entries.
CREATE TABLE memory_relations (
  entry_id         TEXT NOT NULL,
  related_entry_id TEXT NOT NULL,
  relation_kind    TEXT NOT NULL,               -- 'similar' | 'supports' | 'refutes' | 'duplicates'
  score            REAL NOT NULL DEFAULT 0,      -- cosine for 'similar', 1.0 for curated kinds
  created_at       TEXT NOT NULL,
  PRIMARY KEY (entry_id, related_entry_id, relation_kind)
);

-- Daily 5-metric health score per workspace (and optionally per crew).
CREATE TABLE memory_health_snapshots (
  id           TEXT PRIMARY KEY,
  workspace_id TEXT NOT NULL,
  crew_id      TEXT,                            -- NULL = workspace-aggregate
  computed_at  TEXT NOT NULL,
  freshness    REAL NOT NULL,
  coverage     REAL NOT NULL,
  coherence    REAL NOT NULL,
  efficiency   REAL NOT NULL,
  reachability REAL NOT NULL,
  overall      REAL NOT NULL,                   -- weighted sum (see below)
  details      TEXT NOT NULL DEFAULT '{}'
);

​

Scopes

type Scope string
const (
    ScopeOwn        Scope = "own"         // agent's own past only
    ScopeCrewShared Scope = "crew_shared" // own + high-value crew entries
)

​

Hybrid retrieval

hits, err := episodic.HybridRecall(ctx, db, embedder, episodic.Query{
    WorkspaceID: ws,
    AgentID:     agent,
    Scope:       episodic.ScopeOwn,
    QueryText:   "OOM in checkout",
    K:           5,
})

RRF_score(entry) = Σ over rankings r of:  1 / (k_RRF + rank_r(entry))

​

What the sparse leg does with your query text

​

Memory relations (embedding relation graph)

• similar — LinkSimilarOnIndex runs at insert time: for the freshly-indexed entry, query the existing embedding pool, take the top-3 cosine matches above 0.80, and write similar rows. Edges are symmetric — both (entry_id=new, related_entry_id=match) and the reverse (entry_id=match, related_entry_id=new) are inserted with relation_kind='similar', score=cosine, so a cosine relation reads the same from either end.
• supports — LinkSupports is called by the consolidator when it derives a rule from one or more journal entries. The rule entry points (directionally) to every supporting evidence entry: relation_kind='supports', score=1.0. This is what makes consolidated rules traceable back to their evidence.

​

Recall API

hits, err := episodic.Recall(ctx, db, embedder, episodic.Query{
    WorkspaceID: ws,
    CrewID:      crew,
    AgentID:     agent,
    Scope:       episodic.ScopeOwn,
    QueryText:   "production deployment keeper denied",
    K:           5,
})

• WorkspaceID is required.
• ScopeOwn requires AgentID; ScopeCrewShared requires CrewID.
• K caps results (1-50, default 5).

type Hit struct {
    EntryID   string
    Score     float64       // cosine similarity in [0,1], 1=identical
    Age       time.Duration // since original entry's ts
    Summary   string
    EntryType string
    AgentID   string
    Payload   map[string]any
}

​

Importance scoring

importance = BASE × RecencyFactor × (1 + ReferenceBoost/8)

• RecencyFactor(indexed_at, now) = max(0.1, 1 - days/180) — a 0.1 floor keeps old-but-critical memories from going to literal zero
• ReferenceBoost(refs) = log₂(refs + 1) — frequently-recalled entries lift, but the /8 divisor prevents runaway loops from dominating over rare-but-critical ones

​

Untrusted-hints wrapper

​

Embedder

type Embedder interface {
    Embed(ctx context.Context, text string) ([]float32, error)
    Dim() int
    ModelID() string
}

​

Health scoring

• HTTP — GET /api/v1/memory/health[?crew_id=<slug>]. See Memory API.
• CLI — crewship memory health [--crew <slug>]. See crewship memory health.

​

In-session memory nudge

You have 72 new journal entries since your last memory update. Consider appending any recurring pattern you’ve noticed to ~/.memory/AGENT.md before the session ends — the consolidator won’t replace your personal observations.

​

Indexer

err := x.IndexOne(ctx, entry)

​

Gotchas

• Ollama dependency. No Ollama = sparse-only recall (no vector similarity). Set KEEPER_OLLAMA_URL, plus OLLAMA_MODELS="/Volumes/SSD 990 PRO/ollama-models" (external SSD) and ollama serve before ./dev.sh start when testing locally. Check the active mode via GET /healthz (episodic field) or crewship doctor.
• Embedding is best-effort. An embedder error during a sweepOnce pass logs and skips; the entry is retried next sweep.
• Cosine on small sets is fine. Don’t pre-optimise for a vector DB until the per-agent scan latency exceeds 50ms.

​

Related

• Crew Journal — source of entries.
• Consolidate — the nightly workers that produce many of the embeddable entry types (summary.generated, memory.consolidated).