Skip to main content

Where this lives

Integrations (/integrations) — two tabs, because there are exactly two kinds of thing here: Everything else is a section inside a tab, in the left panel — not another top-level tab. Both panels are the same component with different inputs: the same toolbar (search + Filter), the same popover, the same section rows, the same active-filter chips. A page with five top tabs and two differently-behaving rails is not simpler for having more entry points. Under the sections, each panel lists the things themselves — your connections, your connected tool accounts — the way /routines lists routines. A rail that only holds section links makes you open a section to find out what is in it. Search and the Filter popover narrow that list, so what the panel shows is what the main column is showing. Picking one opens its detail: for a connection, which categories it may carry, which agents may post to it, and its own recent deliveries; for a tool account, whose it is, whether its authorisation still works, and which agents can act through it. Those answers used to be spread across a row, an admin-only overview and the delivery log, so nobody could see one connection whole. The Filter popover is the one every faceted page in the app uses (sidebar-kit’s SidebarFilterButton, as on /issues and /routines). Facet counts are computed against the rows the other facets already allow, so a bucket never offers a filter that comes back empty — and a facet never narrows its own counts to zero. Managed tool accounts are not in the Notifications connection list. They were, and it was the most confusing thing on the page: a Discord webhook and a Gmail tool account sat in one table under one word, with different owners, different lifecycles and different meanings of “connected”. It used to live under Settings. That door reads as “my personal preferences”, which is wrong for a set of connections a whole org depends on, and it made the thing an admin most needs — one view of everything this instance is wired into — impossible to build. Settings no longer carries a Notifications row at all (a nav row promises a pane; a row that only bounces you elsewhere does not), but the old links still resolve: /settings?tab=notifications forwards to the connection list and /settings?tab=notification-prefs to the preference matrix. So there is exactly one place rather than two that drift apart. Notifications and Deliveries stay here rather than moving back: the preference matrix is a grid over these connections, and “why didn’t it arrive” is a question about them — splitting either from the list it refers to is the same two-places-for-one-thing this page was built to undo.

Linking to a view

Every tab and section is addressable, so a view can be shared, bookmarked or linked to from elsewhere in the app:
The URL is rewritten as you move, and an unknown or cross-tab section falls back to that tab’s default rather than leaving the panel pointing at a section it does not own.

Adding something: one door

There is no Catalog tab. Browsing what you could connect is a question you ask while adding, not a place you work in, and as a tab it sat at the same weight as the four you actually use. + Add integration asks the kind first — Notifications (somewhere Crewship reaches a person) or Tools & MCP (something an agent can act through) — then the service, then its form. The kinds are a list in the code, so a third one later is one entry rather than a new tab and a second grid to reconcile with the first. Picking Tools jumps straight to that tab’s app catalog, or to the API-key dialog when no key is set yet: browsing a catalog you cannot connect from is a dead end.

Finding a service you have not connected yet

Search on Connections only searches what exists. When it comes back empty but a service matches, the empty state says how many and opens Add integration — searching for telegram before you own a Telegram channel is a question the page can answer rather than a blank list. The service list inside that flow is grouped into Chat, Push, Incident and E-mail & webhook. The grouping comes from the server (GET /api/v1/notification-providers returns a category per provider), so a provider added on the backend appears in the right section without a frontend change. A provider an admin has disabled is shown greyed out and unpickable — visible, and visibly unavailable, rather than missing. Each service carries its own brand mark in its own colour: you recognise Slack by its aubergine long before you read the label, and a column of identical grey glyphs is not something you can scan. The artwork is vendored — CC0 SVG Logos for the full-colour marks, CC0 Simple Icons and MIT Tabler for the monochrome ones, and CC BY-SA Arcticons for Gotify, which exists nowhere under a more permissive licence. A brand whose official colour is black (Matrix) is lifted for display, because rendering it faithfully on a dark-first UI means rendering it invisible.

Tools (MCP)

Composio’s six views are rows in the left panel, not a second tab strip inside the page’s own. Its facets are Toolkit and User — Kind/Status/Scope/Service filter nothing here. The page owns the heading, the KPI row and the actions. The API key sits at the foot of the left panel, showing which key is in use rather than being a button in the action bar — it configures the instance once, and does not belong beside Add integration. Its four reads — inventory, the app catalog, settings and every agent’s bindings — are cached for 30 s with stale-while-revalidate (lib/stale-cache.ts). Leaving and re-entering the tab used to pay the full round trip again and show skeletons for it, including an N+1 fan-out over every agent. A cached surface now paints immediately and refreshes behind; Refresh drops the cache first, so it means refresh. The search box in the left panel drives this tab too — Composio’s own three inner search boxes are suppressed while it is, so one list never has two inputs where only one works. Before a key is saved the panel shows a single Setup row and says the sections appear once there is a key. Six empty sections would be six dead ends.

Where else this shows up

The connections are configured here, but the two questions they raise are asked elsewhere, so the answers appear there too:
  • An agent’s Skills & Tools tab (Crews) — under Integrations, beside Connectors, a Notifications card lists the channels the agent may post to, with Manage access to grant or revoke. It sits there rather than on Overview because that tab is the agent’s access surface, and a channel grant is access. Each pairing is still gated on the channel — ADMIN/OWNER for a workspace channel, ownership for a personal one — so a refusal comes from the server in its own words rather than being guessed at in the UI.
  • A routine’s Overview extends What it touches with Reach through its agents: the routine runs Riley, Riley is granted Gmail, therefore this routine can send mail. That hop is invisible in the DSL, because the grant lives on the agent. Read-only — it is a summary of grants made elsewhere.

Seeing everyone’s connections (admins)

On Connections, ADMIN/OWNER get an Everyone’s connections toggle in the tab row. It widens the list to include other members’ personal channels — but it shows that a member has a channel and of what kind, never where it points: a Telegram chat id is a contact detail, not workspace configuration. Those rows are read-only; only their owner can test, disable or delete them.

Delivery status

The Status column is derived from the delivery log, which is ADMIN/OWNER-only. A member sees Enabled rather than Delivering — the page does not claim to know something it cannot read. Ordering matters: a channel’s most recent failure outranks an older success, and deliveries that were muted or rate-gated never count as delivering, so a silenced channel cannot look healthy.

Two systems, one page

Crewship has grown two outbound-notification mechanisms, documented together here:
  1. Run-terminal broadcast (issue #850, below in Legacy: run-terminal broadcast) — workspace-wide email/webhook channels that fire on a routine run’s completed/failed terminal state. Unchanged by everything below; still the right tool for “tell the whole workspace when this routine fails.”
  2. Category preference system (issue #1412, the rest of this page) — a per-user, Linear/Novu-style category × channel matrix covering routine, issue, agent, system and chat categories, delivered to email, a signed webhook, or any of eleven chat and push destinations, with admin-managed workspace channels AND self-service personal channels, an anti-storm rate gate, and a persistent delivery log.
Both ride the same channel type (notification_channels) and the same low-level delivery code — a channel you create can serve BOTH systems (a workspace webhook can both receive the legacy run.completed broadcast and be selected in someone’s category matrix).
Workspace-scoped channel writes require ADMIN/OWNER. A personal channel (--personal) is self-service — any member may add their own. A webhook signing secret is shown once at creation — store it then; it can never be read back. Chat destinations have nothing to reveal: their delivery URL is composed from the values you supplied.

Categories

Categories are grouped so the matrix stays readable. Every one of them has a real event behind it — a category you can switch on always delivers something.

Routines

Issues

Agents

System

Chat & memory

agents.approval and agents.escalation always deliver immediately and skip the anti-storm token bucket entirely — a blocking HITL item must never be dropped as “too many notifications.”
Renamed in taxonomy v2. The original vocabulary used approvals, escalations, runs.failed, runs.completed, budget and system. Four of the nine categories it defined had no event behind them at all — they were switchable rows that could never deliver. Your existing preferences and per-channel allowlists were migrated automatically: an opted-in cell stays opted in, a muted channel stays muted, and system was split across both system.health and system.migration rather than dropping half of what it meant. Only the delivery log keeps the old names, because rewriting history would misreport what was actually sent.
Every cell’s state is off (default) or immediate. A digest value is legal in the schema (so a future digest-batching window needs no migration) but nothing writes it yet — that’s v2 scope.

Channel types

Webhook (signed)

A webhook channel POSTs a JSON payload to your URL and signs it so you can verify the request really came from your Crewship instance.
Headers: A category-routed message (the #1412 system — an approval, a failed run, a chat reply) uses a different payload on the same channel, signed the same way and carrying X-Crewship-Category instead of X-Crewship-Event:
url is the primary link — the first entry of links — and keeps the meaning it had before links existed, so a receiver already reading it does not break. vars carries the source event’s own facts; a producer that starts recording a new fact surfaces it here without a schema change, so treat the object as open. Notifications carry links to the thing they are about, so a message in Discord is something you can act on rather than something you have to go look up. Set CREWSHIP_PUBLIC_URL to the address people reach this instance at:
Without it, links are delivered as app-relative paths (/issues/CS-12). Those work in the in-app inbox and are useless in a chat client, which has no origin to resolve them against. Crewship will not guess a hostname — a link that looks right and 404s costs more than an obviously relative one. Titles are sent natively. Every chat and push service in the catalog except Google Chat has its own title field, and Crewship uses it rather than gluing the title onto the front of the message text. On a chat service that is the difference between a bold header and a plain first line; on a push service — Pushover, ntfy, Gotify — the title is the line your phone shows on the lock screen, so this is the difference between a useful notification and one you have to open to identify. Google Chat discards the field, so there the title stays in the text and nothing is lost. Discord arrives as one message. shoutrrr’s Discord service defaults to sending each line of a message as its own embedded item, which turns an ordinary notification into a stack of disconnected boxes — title in one, each body line in another, the link in a third. Crewship overrides that on the way out, so channels created before this change are fixed too without being re-added. A message still splits on Discord’s own 2000-character limit. Links point only where a page actually resolves. An issue, a mission timeline, a chat session and an agent’s approvals have their own routes and are linked precisely; a failed run gets the runs list, because there is no per-run page to send you to yet. No notification links to an invented filter that the destination silently ignores. Verify it (Go):
Delivery is best-effort with retries: 3 attempts with exponential backoff. A 5xx or 429 is retried; a 4xx (other than 429) is treated as a permanent client error and dropped. A failing channel is logged — it never fails the run that triggered it.

Email

An email channel sends via the instance mailer. Email delivery must be configured (RESEND_API_KEY and RESEND_FROM set on the server) — otherwise creating an email channel is rejected with a clear error, so you learn at configuration time rather than silently losing notifications (fail-closed).

Chat and push

A chat channel delivers to Discord, Slack, Telegram, ntfy, Gotify, Pushover, Mattermost, Matrix, Microsoft Teams, Google Chat, or Opsgenie. Each destination carries a categorychat, push or incident — which is the one piece of routing advice the list can give you: chat posts into a room somebody is expected to be reading, push reaches one person’s device now, and incident escalates through an on-call rota until someone acknowledges. It is a column in crewship notifychannel providers and the section heading in the dashboard catalog; both read the same field from the server. Each one asks for what that service actually gives you — Discord for the webhook URL you copy out of Server Settings, Telegram for a bot token and a chat ID — with a line under each field saying where to find it. You never construct or see a delivery URL: the server composes one from your answers and stores it encrypted, in the same secret_enc column the webhook signing secret uses. It is never returned by any read endpoint. To see what a provider needs:
Then fill the fields in with --field key=value:
Test before you save. crewship notifychannel test-draft (and the Send test button on the form) sends one real notification using the values you have typed and stores nothing. Use it to confirm a token is right instead of saving a channel and discovering the mistake when a real alert goes missing.
A chat channel also participates in the legacy run-terminal broadcast if it subscribes to events — chat and push are new destinations alongside email/webhook, not a replacement. An instance admin (ADMIN or OWNER) can disable a provider instance-wide:
A disabled provider is rejected at channel-create time (fail-closed). It does not stop channels that already exist — they keep delivering. To silence one of those, disable or remove that channel. If you need a true kill switch for a provider, disable it and the channels already on it.
--type shoutrrr still works as a hidden alias for --type chat, and --url still accepts a pre-composed delivery URL, so existing scripts and backup restores keep working. Both are undocumented on purpose: nobody should have to learn a URL syntax to connect Discord.

Agents sending notifications

An agent can send a notification itself — “the migration finished”, “this needs a decision”, “the build is broken” — through the notify_send tool inside its container. An agent has no channels by default. Access is granted per (channel, agent) by a human, and there is no way for an agent to grant itself one. That is deliberate: an agent can already reach the network, so the question this answers is not “can it send” but which of your team’s channels may it speak on. On Slack or Discord a reader cannot tell an agent’s message from a colleague’s at a glance, so an agent that could post anywhere by virtue of existing turns one confused or prompt-injected agent into a workspace-wide megaphone.
Pairing takes the same authority as editing the channel: ADMIN/OWNER for a workspace channel, ownership for a personal one.

What the agent sees

Two tools, injected automatically into every agent’s MCP config:

What is enforced on the way out

Every agent send lands on the Activity timeline as notification.delivered with the agent as the actor, so “who sent this to Slack?” has an answer.
Agent messages are not routed through the per-user preference matrix. The pairing is the authorization — a human already decided this agent may post here — so letting someone’s mute silently swallow it would make an approved grant look broken from both ends.

Changing what a notification says

A routine’s notify step has always written its own message. Everything Crewship generates itself — “Pipeline nightly completed”, “Scheduled routine failed: deploy” — had its wording computed in Go, one string per producer, with no way to change it. Templates are the override.
Two namespaces are available, and the syntax is the same {{ … }} a routine step uses, so there is nothing new to learn: Nothing is lost when a reference misses. A fact this particular event does not carry renders empty, and a template that renders to nothing falls back to the producer’s own wording — so a notification never arrives with a blank subject line, which on a push service is a blank entry on someone’s lock screen. Scope is the category, optionally narrowed to one channel with --channel. Wording is a property of the event: “a routine failed” reads the same wherever it goes. The narrowing exists for the case where one destination genuinely wants something different — a terse line for a pager, a fuller one for e-mail — and a channel-specific template wins over the all-channels one. Templates never rewrite somebody else’s words. A routine’s notify step and an agent’s message carry an author’s text, and an operator’s category template does not touch them. Those already say what their author chose. Setting a template is ADMIN+: a preference is “do I want to hear about this”, a template is “what everyone sees”. Clearing both fields removes the override and restores the shipped wording. An unknown category, or a reference to a namespace that does not exist, is rejected when you save it rather than when a notification fails to arrive.

Payload safety

Every author-influenced field of a notification — title, body, link labels and every value in vars, including nested ones — is passed through the secret scrubber (the same redaction the notify step uses) before it leaves the instance. Bodies are additionally capped. API keys and tokens in a run’s output, a journal summary or an agent’s reply never reach a webhook body, an email, or a chat message. Redaction is a property of delivering a message, applied once for every producer. It is not something a caller opts into: a producer that starts carrying a new fact is covered by default rather than when someone remembers.

Two-layer preferences (the #1412 category system)

Notification routing follows the Linear/Novu pattern: an admin layer decides what’s available, a user layer decides what reaches them.

Admin layer

An ADMIN/OWNER controls, per workspace channel:
  • Which providers are enabled instance-wide (crewship notifychannel providers) — a create-time gate, not a kill switch for existing channels.
  • Which categories a channel may serve (--categories, PATCH .../notification-channels/{id}) — empty means “every category.”
  • A priority floor (--min-priority) — an item below the floor never reaches that channel, however the user’s matrix is set.

User layer

Every member manages their own matrix — a category (row) × channel (column) grid, off/immediate per cell — over workspace channels the admin allowlisted for that category plus their own personal channels. A cell defaults to off: nothing is delivered externally until a member opts in. A special * “category” mutes a channel entirely, overriding every other cell for that channel — the fastest way to say “stop pinging me on Telegram” without touching nine individual cells.

Presence-aware suppression

For chat.replies, a member watching the chat live (an open WebSocket subscription on that session) never gets an external push for the same reply that’s streaming to their screen — the same presence gate the in-product chat-reply bell already uses.

Anti-storm rate limiting

Delivery is gated by an in-process token bucket per (recipient, channel, category) — a burst of 5, refilling at roughly 1 token per 30 seconds by default — so a misconfigured routine notifying the same person in a loop can’t flood their Slack. Overflow is a soft drop: it’s recorded in the delivery log with status dropped_rate, never silently swallowed. agents.approval and agents.escalation bypass the rate gate entirely — a blocking human-in-the-loop item is never dropped for being “too frequent.” Repeated fires for the exact same source event (a retried hook, a re-processed job) coalesce on category:source_id — one delivery row, one send, regardless of how many times the underlying write-through fires.

Delivery log

Every attempted delivery through the #1412 category system is recorded — pending → sent | failed | dropped_pref | dropped_rate — so “why didn’t my notification arrive?” has an answer instead of a buried log line.
Reading the log is admin-only (GET /api/v1/notification-deliveries, ADMIN/OWNER) since it spans every recipient in the workspace, not just the caller. The Deliveries tab on /integrations renders the same rows, with each status translated into a reason — dropped_pref reads as recipient has this category muted, dropped_rate as rate limit overflow; not delivered — because the whole point is telling a quiet channel apart from a broken one.
The API serves this log with snake_case keys (channel_id, created_at, …). Until 2026-07 the fields carried no JSON tags at all, so the server emitted Go field names and any client asking for channel_id got nothing: crewship notifychannel deliveries printed blank CHANNEL, USER and CREATED columns. If you are scripting against an older instance, that is why.

On the Activity timeline

Every outbound send also lands on the Activity timeline with its own webhook icon, so “this left the instance and went to Slack” sits next to the event that caused it rather than only in the admin-only log above:
These entries never themselves trigger a notification. Delivery records commit through the same journal the notification router watches, so routing one would notify about having notified — and that notification would emit another record. The mapping refuses them explicitly.

CLI

Provide your own webhook secret with --secret <value> instead of letting Crewship generate one. notifychannel add/list/providers/deliveries and notify prefs get honour the global --format flag (json/yaml/ndjson) for scripting — see Output Formats.

API

All routes are workspace-scoped (the workspace comes from your auth context).
A workspace-scoped channel write (the default; personal omitted or false) requires ADMIN/OWNER — tightened from MANAGER+ as part of #1412, since a workspace channel is now a shared delivery target for the whole category-preference system, not just a run-terminal broadcast target. A personal channel (personal: true) is self-service: any authenticated member may create their own, and only its owner (never another member, regardless of role) may update, test, or delete it. owner_user_id is always forced to the authenticated caller server-side — never trust a client-supplied owner.

Endpoint contracts

The following route contracts are the HTTP boundary for the table above. All requests require authentication and workspace context unless stated otherwise.

GET /api/v1/notification-channels

Request: optional list filters; no body. Response: 200 OK with workspace channels and redacted secrets. Statuses: 200 OK; 401 Unauthorized without authentication and 403 Forbidden without workspace access.

POST /api/v1/notification-channels

Request: JSON channel definition; personal may opt into the self-service path described above. Response: 201 Created with the created channel; webhook secret is returned once. Statuses: 201 Created; 400 Bad Request for invalid channel data and 403 Forbidden when the caller lacks the required role.

PATCH /api/v1/notification-channels/

Request: id path parameter and a JSON patch for channel settings. Response: 200 OK with the updated channel. Statuses: 200 OK; 400 Bad Request, 403 Forbidden, or 404 Not Found for invalid input, insufficient ownership/role, or an unavailable channel.

POST /api/v1/notification-channels//test

Request: id path parameter; no body required. Response: 200 OK with the test result. Statuses: 200 OK; 403 Forbidden or 404 Not Found when the caller cannot test the channel or it is unavailable.

DELETE /api/v1/notification-channels/

Request: id path parameter; no body. Response: JSON { "deleted": "<id>" }. Statuses: 200 OK; 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, or 500 Internal Server Error for invalid input, missing workspace, insufficient ownership/role, an unavailable channel, or a delete failure.

GET /api/v1/notification-channels//agents

Request: id path parameter; no body. Response: 200 OK with the agents allowed to send to the channel. Statuses: 200 OK; 403 Forbidden or 404 Not Found when the caller cannot inspect the channel.

POST /api/v1/notification-channels//agents

Request: id path parameter and JSON { "agent_id": "…" }. Response: 200 OK with { "channel_id": "…", "agent_id": "…", "allowed": true }. Statuses: 200 OK; 400 Bad Request for invalid input or an agent outside the workspace, 403 Forbidden for insufficient role, and 404 Not Found for an unavailable channel.

DELETE /api/v1/notification-channels//agents/

Request: id and agentId path parameters; no body. Response: 200 OK with { "channel_id": "…", "agent_id": "…", "allowed": false }. Statuses: 200 OK; 403 Forbidden when the caller cannot change the channel, and 404 Not Found when the relationship is absent or the channel is unavailable.

POST /api/v1/notification-channels/test

Request: JSON unsaved channel definition. Response: 200 OK with the provider test result; nothing is persisted. Statuses: 200 OK; 400 Bad Request for invalid draft data and 403 Forbidden when the caller lacks access to the workspace.

GET /api/v1/notification-providers

Request: no body. Response: 200 OK with provider metadata, form fields, and enabled state. Statuses: 200 OK; 401 Unauthorized without authentication.

PATCH /api/v1/notification-providers/

Request: provider path parameter and JSON provider settings. Response: 200 OK with the updated provider state. Statuses: 200 OK; 400 Bad Request, 403 Forbidden, or 404 Not Found for invalid input, insufficient role, or an unknown provider.

GET /api/v1/me/notification-prefs

Request: no body; the authenticated user selects the preference owner. Response: 200 OK with that user’s category-by-channel matrix. Statuses: 200 OK; 401 Unauthorized without authentication.

PUT /api/v1/me/notification-prefs

Request: JSON { "cells": [{ "category", "channel_id", "state" }] }; the submitted cells are upserted. Response: 200 OK with the updated preference result. Statuses: 200 OK; 400 Bad Request for invalid cells and 401 Unauthorized without authentication.

GET /api/v1/notification-deliveries

Request: optional status, channel_id, category, and limit query parameters; no body. Response: 200 OK with the delivery log. Statuses: 200 OK; 403 Forbidden unless the caller is ADMIN/OWNER.

GET /api/v1/notification-templates

Request: no body. Response: 200 OK with workspace wording overrides. Statuses: 200 OK; 403 Forbidden unless the caller is ADMIN/OWNER.

PUT /api/v1/notification-templates

Request: JSON wording override containing a category, title, and body. Response: 200 OK with the saved override. Statuses: 200 OK; 400 Bad Request for an invalid category/body and 403 Forbidden unless the caller is ADMIN/OWNER.

DELETE /api/v1/notification-templates

Request: required category and optional channel_id query parameters; no body. Response: 204 No Content. Statuses: 204 No Content; 400 Bad Request for invalid query parameters and 403 Forbidden unless the caller is ADMIN/OWNER. Create body:
events (optional, legacy broadcast — defaults to ["run.failed"]; accepts completed, failed, or all), categories (optional, admin allowlist — defaults to every category), min_priority (optional — defaults to low) are independent of each other: a channel can serve the legacy broadcast, the category matrix, both, or neither. The create response for a webhook channel includes the secret field once — the signing secret — the only time it is ever returned. Preference matrix body (PUT /api/v1/me/notification-prefs):
PUT upserts the cells you send — it does not replace your whole matrix, so a single-cell UI edit never needs to resend every other cell.

Legacy: run-terminal broadcast

The original (#850) mechanism: workspace-wide email/webhook/chat channels that fire on a routine run’s terminal state. Unchanged by #1412 — this is documented for reference, not deprecated; it’s still the right tool for “tell the whole workspace when this routine fails,” independent of anyone’s personal preference matrix. Each channel subscribes to the outcomes it cares about via events — by default failures only, so a routine that runs hourly doesn’t flood an inbox with success pings. Opt into completions with --events completed (or --events all). See Channel types above for the webhook payload shape and HMAC verification. Delivery is hooked on the run’s terminal write in the pipeline finalize path — not on the CLI — so scheduled runs with no connected client notify exactly the same as interactive ones. Only completed and failed outcomes fire (and then only to channels subscribed to that outcome); cancelled and interrupted are operational states and are intentionally silent. Fan-out is asynchronous and best-effort, so it never slows or fails the run — a failing channel is logged, not retried into oblivion (3 attempts with exponential backoff, then it drops).