​

Episodic Memory

​

Embedding criteria

// internal/episodic/types.go
func shouldEmbed(entryType string, severity string) bool {
    switch entryType {
    case "peer.escalation",
         "summary.generated",
         "memory.consolidated",
         "approval.denied",
         "eval.regression_detected":
        return true
    case "keeper.decision":
        return severity == "warn" || severity == "error"
    case "mission.status_change":
        return severity == "warn" || severity == "error"
    }
    return false
}

​

Storage

CREATE TABLE journal_embeddings (
  entry_id           TEXT PRIMARY KEY REFERENCES journal_entries(id) ON DELETE CASCADE,
  vector             BLOB NOT NULL,     -- raw float32 bytes
  dim                INTEGER NOT NULL,
  model              TEXT NOT NULL,     -- "nomic-embed-text", etc.
  indexed_at         TEXT NOT NULL,
  importance_score   REAL DEFAULT 0.5,  -- fed into recall sort (see "Importance scoring")
  reference_count    INTEGER DEFAULT 0, -- bumped when a hit lands in a prompt
  last_referenced_at TEXT               -- stamped at the same time
);

​

Migration 55 additions

-- FTS5 contentless mirror of journal_entries.summary + payload.
-- Insert/update/delete triggers keep it in lockstep with the base table.
-- Powers the ?q= parameter on /api/v1/journal and the BM25 leg of HybridRecall.
CREATE VIRTUAL TABLE journal_entries_fts USING fts5(
  summary, payload, content='journal_entries', content_rowid='rowid'
);

-- Compaction sink. Aged rows from journal_entries land here with payload
-- truncated to 400 chars before being deleted from the base table.
CREATE TABLE journal_entries_archived (
  id           TEXT PRIMARY KEY,
  workspace_id TEXT NOT NULL,
  ts           TEXT NOT NULL,
  entry_type   TEXT NOT NULL,
  severity     TEXT NOT NULL,
  summary      TEXT NOT NULL,
  payload      TEXT NOT NULL,                 -- 400-char truncation
  archived_at  TEXT NOT NULL
);

-- embedding relation graph: similarity edges linking related embeddings.
CREATE TABLE memory_relations (
  src_entry_id TEXT NOT NULL,
  dst_entry_id TEXT NOT NULL,
  relation     TEXT NOT NULL,                 -- 'similar' | 'supports' | 'contradicts'
  weight       REAL NOT NULL DEFAULT 0.0,     -- cosine for 'similar', 1.0 for hand-asserted
  created_at   TEXT NOT NULL,
  PRIMARY KEY (src_entry_id, dst_entry_id, relation)
);

-- Daily 5-metric health score per workspace (and optionally per crew).
CREATE TABLE memory_health_snapshots (
  workspace_id   TEXT NOT NULL,
  crew_id        TEXT,                        -- NULL = workspace-aggregate
  snapshot_date  TEXT NOT NULL,               -- YYYY-MM-DD UTC
  freshness      REAL NOT NULL,
  coverage       REAL NOT NULL,
  coherence      REAL NOT NULL,
  efficiency     REAL NOT NULL,
  reachability   REAL NOT NULL,
  score          REAL NOT NULL,               -- weighted sum (see below)
  PRIMARY KEY (workspace_id, crew_id, snapshot_date)
);

​

Scopes

type Scope string
const (
    ScopeOwn        Scope = "own"         // agent's own past only
    ScopeCrewShared Scope = "crew_shared" // own + high-value crew entries
)

​

Hybrid retrieval

hits, err := episodic.HybridRecall(ctx, db, embedder, episodic.Query{
    WorkspaceID: ws,
    AgentID:     agent,
    Scope:       episodic.ScopeOwn,
    QueryText:   "OOM in checkout",
    K:           5,
})

RRF_score(entry) = Σ over rankings r of:  1 / (k_RRF + rank_r(entry))

​

Memory relations (embedding relation graph)

• similar — LinkSimilarOnIndex runs at insert time: for the freshly-indexed entry, query the existing embedding pool, take the top-3 cosine matches above 0.80, write (src=new, dst=match, relation='similar', weight=cosine) rows. Edges are unidirectional (the new entry points to its predecessors); the inverse direction is implied at read time.
• supports — LinkSupports is called by the consolidator when it derives a rule from one or more journal entries. The rule entry points to every supporting evidence entry: relation='supports', weight=1.0. This is what makes consolidated rules traceable back to their evidence.

​

Recall API

hits, err := episodic.Recall(ctx, db, embedder, episodic.Query{
    WorkspaceID: ws,
    CrewID:      crew,
    AgentID:     agent,
    Scope:       episodic.ScopeOwn,
    QueryText:   "production deployment keeper denied",
    K:           5,
})

• WorkspaceID is required.
• ScopeOwn requires AgentID; ScopeCrewShared requires CrewID.
• K caps results (1-50, default 5).

type Hit struct {
    EntryID   string
    Score     float64       // cosine similarity in [0,1], 1=identical
    Age       time.Duration // since original entry's ts
    Summary   string
    EntryType string
    AgentID   string
    Payload   map[string]any
}

​

Importance scoring

importance = BASE × RecencyFactor × (1 + ReferenceBoost/8)

• RecencyFactor(indexed_at, now) = max(0.1, 1 - days/180) — a 0.1 floor keeps old-but-critical memories from going to literal zero
• ReferenceBoost(refs) = log₂(refs + 1) — frequently-recalled entries lift, but the /8 divisor prevents runaway loops from dominating over rare-but-critical ones

​

Untrusted-hints wrapper

​

Embedder

type Embedder interface {
    Embed(ctx context.Context, text string) ([]float32, error)
    Dim() int
    ModelID() string
}

​

Health scoring

• HTTP — GET /api/v1/memory/health[?crew_id=<slug>]. See Memory API.
• CLI — crewship memory health [--crew <slug>]. See crewship memory health.

​

In-session memory nudge

You have 72 new journal entries since your last memory update. Consider appending any recurring pattern you’ve noticed to ~/.memory/AGENT.md before the session ends — the consolidator won’t replace your personal observations.

​

Indexer

err := x.IndexOne(ctx, entry)

​

Gotchas

• Ollama dependency. No Ollama = sparse-only recall (no vector similarity). Set KEEPER_OLLAMA_URL, plus OLLAMA_MODELS="/Volumes/SSD 990 PRO/ollama-models" (external SSD) and ollama serve before ./dev.sh start when testing locally. Check the active mode via GET /healthz (episodic field) or crewship doctor.
• Embedding is best-effort. An embedder error during a sweepOnce pass logs and skips; the entry is retried next sweep.
• Cosine on small sets is fine. Don’t pre-optimise for a vector DB until the per-agent scan latency exceeds 50ms.

​

Related

• Crew Journal — source of entries.
• Consolidate — the nightly workers that produce many of the embeddable entry types (summary.generated, memory.consolidated).