​

Memory Provider interface

​

Why a formal interface

• The dispatcher remains the on-disk path (call sites unchanged)
• New backends implement four methods (Retain / Recall / Forget / Health) against the interface
• The orchestrator picks a Provider at boot from config — no per-call branching needed
• Tests can stub the interface without spinning up a real backend

​

The four methods

type Provider interface {
    Retain(ctx, RetainRequest) (RetainResult, error)
    Recall(ctx, RecallRequest) (RecallResult, error)
    Forget(ctx, ForgetRequest) (ForgetResult, error)
    Health(ctx) HealthStatus
}

​

Retain — persist content

type RetainRequest struct {
    WorkspaceID string
    AgentID     string // optional
    CrewID      string // optional
    Tier        string // AGENT | CREW | PERSONA | pins | daily | peers | lessons
    Key         string // tier-specific (date for daily, slug for peers)
    Content     string // UTF-8 body
    Mode        string // "replace" | "append"
}

type RetainResult struct {
    ID    string // canonical handle for later Forget(byID)
    Bytes int    // size persisted (useful for cap-aware callers)
}

• Flat struct, no nested option bags. New fields require deciding which bag they live in; flat ages better.
• Mode is required, no default. A “default mode” hides intent at the wire — when the operator inspects logs they should see exactly what was requested. Local impl returns an error for an empty mode; remote impls should too.
• Tier is the closed enum from the dispatcher. Adding a tier requires extending validTiers in tools.go AND every provider that switches on tier — a tier nobody implements is worse than a tier nobody declares.
• Key is tier-specific. Required for daily (date) and peers (user slug); ignored elsewhere. Providers validate at the boundary; the local impl errors on missing key for these tiers.
• Content is plain UTF-8. Binary payloads use the existing blob-store path, not the memory dispatcher.

​

Recall — search snippets

type RecallRequest struct {
    WorkspaceID string
    AgentID     string // optional: limit to a single agent's tiers
    CrewID      string // optional: limit to crew-shared tiers
    Tier        string // optional: empty = all accessible tiers
    Query       string // search text
    Limit       int    // advisory; provider SHOULD cap at 20
}

type RecallResult struct {
    Hits        []RecallSnippet `json:"hits"`        // always present (may be empty)
    Quarantined []string        `json:"quarantined,omitempty"` // file labels the scanner rejected; surface to operator UI but NEVER feed back to the model
}

type RecallSnippet struct {
    Source  string  `json:"source"`            // tier-relative label (NEVER an absolute filesystem path)
    Snippet string  `json:"snippet"`
    Line    int     `json:"line,omitempty"`
    Score   float64 `json:"score,omitempty"`   // 0..1; provider-specific (local impl returns 1.0 always)
}

• Tier empty means “all accessible tiers” — providers iterate the tiers the caller’s AgentID can read.
• Limit is advisory. Providers SHOULD return fewer than Limit when the corpus is small; they SHOULD cap at 20 to match the dispatcher’s existing JSON Schema (the model has been trained on the cap).
• Score is 0..1 and provider-specific. Local impl returns 1.0 for substring hits (no ranking signal). Vector providers return cosine similarity. BM25 providers return rescaled BM25. Don’t compare scores across providers.
• Snippet is the matched excerpt the model sees in the tool result. Keep it short (≤500 chars) — the model has the Source if it wants to fetch the full content via memory.read.
• Source is the tier-relative label, never an absolute filesystem path (leaking bind-mount layout to the model is a small but real info-disclosure surface — see tools.go::pathToSourceLabel for the canonicalisation contract).
• Quarantined is for files the inbound prompt-injection scanner rejected (Layer 5 — memory prompt-injection scanner). Surface these in the operator UI but NEVER feed them back to the model — the model would re-read poisoned content into its context.

​

Forget — delete

type ForgetRequest struct {
    WorkspaceID   string
    ID            string // canonical id from RetainResult (per-item)
    DataSubjectID string // GDPR cascade selector (per-subject)
    // Exactly one of ID / DataSubjectID MUST be non-empty:
    //   - ID set, DataSubjectID empty  = per-id delete
    //   - DataSubjectID set, ID empty  = cascade delete for SAR
    //   - both empty                    = error (no-op deletes are caller bugs)
    //   - both set                      = error (intent ambiguity)
}

type ForgetResult struct {
    Removed int // canonical count for the caller's audit log; 0 + nil error = no-op (re-issued DELETE for an already-deleted item is fine)
}

• Two selectors, mutually exclusive. ID is for the operator-driven “delete this specific row” path. DataSubjectID is the GDPR Article 17 cascade path (PR-F1). Mixing them is rejected — the caller knows which they want, and round-9 hardening rejects the both-set case explicitly with a “must set exactly one” error.
• The on-disk LocalDispatcher rejects DataSubjectID with an explicit “cascade not implemented in local provider; use the PR-F1 API endpoint” error because the on-disk tier doesn’t carry data_subject_id foreign keys. GDPR cascade goes through the SQL handler (internal/api/admin_gdpr.go) which queries the cascade-aware tables directly. This is documented behaviour — local-disk providers should return the same error.
• Removed reports items actually removed. Already-absent items count as 0 (per-id) or contribute to the total based on what the provider actually touched. Re-issued DELETE for an already-deleted item returns Removed: 0, error: nil (no-op semantics, not a failure).

​

Health — liveness check

type HealthStatus struct {
    OK        bool      `json:"ok"`
    Message   string    `json:"message,omitempty"`     // human-readable when OK=false
    CheckedAt time.Time `json:"checked_at"`            // server-side timestamp of the check
}

• Health MUST return promptly — under 200ms for local, under 1000ms for remote. A stuck Health() blocks the operator’s aux-status panel render. Implementations SHOULD use a hard deadline (context.WithTimeout(ctx, 1*time.Second) internally) and return OK=false, Message="health check timeout" rather than block.
• Message is for failure context only. On OK=true, leave it empty. On OK=false, give enough text for the operator to triage (“connection refused”, “auth expired”, “503 from upstream”).
• CheckedAt lets the panel show “last successful check N seconds ago” without piggy-backing on the framework’s clock — each provider stamps its own observation time.
• A provider identifier (e.g. "local-dispatcher", "http://memory.internal:9090") lives in the workspace’s provider config, NOT on the HealthStatus — the panel knows which provider it’s asking about.

​

Reference implementation — LocalDispatcher

type LocalDispatcher struct {
    d *Dispatcher // the existing on-disk dispatcher from tools.go
}

func NewLocalDispatcher(ac AgentContext) *LocalDispatcher {
    return &LocalDispatcher{d: NewDispatcher(ac)}
}

1. It proves the Provider interface fits the existing dispatcher logic without forcing a rewrite — if the on-disk impl didn’t fit, we’d need to widen the interface
2. It gives tests a stub seam — code that depends on Provider can use LocalDispatcher in unit tests without spinning up a remote backend, and remote-provider tests can use the same dispatcher fixtures the local one uses

​

Building a new provider

​

1. Define the type

// internal/memory/providers/foo/provider.go
package fooprovider

type Provider struct {
    httpClient *http.Client
    baseURL    string
    apiKey     string
}

func New(cfg Config) (*Provider, error) {
    // validate cfg, return error on bad config
}

​

2. Implement the four methods

• Wrap upstream errors so callers can errors.Is(err, ErrUpstreamTimeout) — don’t leak raw HTTP error strings
• Implement timeouts via the passed ctx; don’t trust upstream to time out cooperatively
• Return OK=false from Health rather than panicking on a network blip — the operator can see the panel and react

​

3. Wire into the bootstrap

• crewship.yaml config block (e.g. memory.provider: foo + memory.foo.url: etc.)
• A factory in the bootstrap that constructs the provider from config
• A test that exercises Retain → Recall → Forget round-trip against a mock upstream

​

4. Run the parity test suite

​

5. Document the operator-facing behaviour

• A line in Agent memory under “Backends” naming the new option
• A config block reference in Configuration → Providers
• A threat-model note if the provider changes any tenant-isolation or data-locality property

​

Versioning

​

When NOT to add a provider

• The data needs to live in a backend that has its own SLA, security perimeter, or query model (vector recall, conversation history, GDPR-tagged enterprise store)
• The operator already runs that backend for other reasons and wants Crewship’s agents to share it

• You want to add a new memory tier alongside AGENT / CREW / PERSONA / etc. — that’s a validTiers extension in the dispatcher, not a provider
• You want to add a new memory tool alongside memory.read / memory.write / etc. — that’s a ToolSchemas addition in the dispatcher
• You want the local backend to behave slightly differently — that’s a fork of LocalDispatcher not a new provider

​

Cross-references

• internal/memory/provider.go — the interface + reference impl
• internal/memory/provider_test.go — the test suite shape
• Agent memory — operator-facing memory model
• Configuration → Providers — where providers get wired
• GDPR cascade — how Forget interacts with Article 17