Skip to content

feat(channel): migrate Intelligence channel host to CopilotKit v2 managed-channels (channels 0.2) - #10

Open
jerelvelarde wants to merge 50 commits into
mainfrom
jerel/opentag-intelligence-v2
Open

feat(channel): migrate Intelligence channel host to CopilotKit v2 managed-channels (channels 0.2)#10
jerelvelarde wants to merge 50 commits into
mainfrom
jerel/opentag-intelligence-v2

Conversation

@jerelvelarde

@jerelvelarde jerelvelarde commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

Migrates OpenTag's Intelligence channel host (app/managed.ts, pnpm channel) from the deprecated @copilotkit/channels-intelligence gateway launcher to the v2 managed-channels runtime, so it connects to a current CopilotKit Intelligence project (opentag-dev / channel kite-opentag).

The live Intelligence dashboard documents a runtime shape OpenTag didn't ship: @copilotkit/runtime/v2 (CopilotRuntime, CopilotKitIntelligence) + @copilotkit/channels (createChannel) + @copilotkit/runtime/v2/node (createCopilotNodeListener), with the channel declared by name and env reduced to INTELLIGENCE_API_URL + INTELLIGENCE_GATEWAY_WS_URL + INTELLIGENCE_API_KEY — no org/project/channel IDs (the runtime derives them from the credentials + channel name).

The SDK typings back this shape: createChannel({ provider }) is documented as "the managed delivery provider a no-adapter Channel targets when activated via CopilotKit Intelligence." This PR builds exactly that — a channel with no adapters, activated when the node listener mounts.

⚠️ Overlaps #8 — read before merging

#8 (feat/intelligence-adapter) is a second, independent implementation of this same migration. Both bump @copilotkit/channels* to 0.2.1 and rewrite app/managed.ts. They conflict in 22 files with each other; #8 conflicts in ~15 with main.

#8 branches from df93bc0, which predates both merged PRs #6 (Intelligence gateway + Python agent/) and #7 (Railway IaC) — so it has no knowledge of agent/, .railway/, or the pnpm channel / pnpm agent scripts, and its package.json deletes the latter two.

#8 this PR
Shape createChannel({ adapters: [intelligenceAdapter()] }) no-adapter channel → CopilotRuntime({ channels })
Transport HTTP poll loop, channel.start() gateway WS, activated by the node listener
Env COPILOTKIT_INTELLIGENCE_URL, COPILOTKIT_API_KEY INTELLIGENCE_API_URL / _GATEWAY_WS_URL / _API_KEY
Platforms Slack + Teams Slack (provider defaults to slack)
Base df93bc0 (pre-#6, pre-#7) current main
Verified Slack + Teams end-to-end typecheck + 122 unit tests; no live run yet

Both APIs still exist in 0.2.1, so neither is wrong — but #8's own MANAGED.md documents it against a feature-flagged Intelligence branch (codex/hosted-bots-infra-vertical-slice, FF_MANAGED_BOTS=true), and the SDK calls that path "V1 Intelligence Channel exclusivity."

#8 has real end-to-end Teams + Slack verification that this PR does not, and that's the one thing that can't be cherry-picked. Worth a conversation with @AlemTuzlak before either lands. Suggested split: land this as the base, then port #8's genuinely better pieces (listed at the bottom).

What changed

Dependencies

  • @copilotkit/channels* ^0.1.1^0.2.1; @copilotkit/runtime floor ^1.62.3^1.63.2 (where the v2 channels API + createCopilotNodeListener ship).
  • Dropped the now-redundant direct @copilotkit/channels-intelligence dep (still present transitively).

Code

  • app/** migrated to the 0.2 API — mechanical rename: createBotcreateChannel, BotToolChannelTool, BotCommandChannelCommand, defineBotTooldefineChannelTool, defineBotCommanddefineChannelCommand, BotChannel, BotNodeChannelNode. Behavior unchanged.
  • app/index.ts (self-hosted) migrated to createChannel; per-platform gating / handlers / shutdown preserved.
  • app/managed.ts rewritten to the v2 managed-channels runtime: createChannel({ name: "kite-opentag" })new CopilotRuntime({ agents, intelligence, identifyUser, channels })createCopilotNodeListener; awaits listener.channels.ready() (fail-loud, exit non-zero on activation error), adds a bind-error handler, and normalizes blank env.

Config / docs

  • .env.example, .railway/railway.ts, README.md, setup.md updated to the v2 var set + runtime model.
  • .railway/railway.ts: INTELLIGENCE_API_URL/_GATEWAY_WS_URL are now preserve() (deployer-set, not clobbered by railway config apply); the channel service installs Chromium (playwright install --with-deps chromium + PLAYWRIGHT_BROWSERS_PATH=0 + runtime apt libs) so inline chart/diagram rendering works on deploy; .railway/** added to check-types.

Suggested review order

  1. app/managed.ts — the only genuinely new file. Everything else is either mechanical rename or config.
  2. app/managed.test.ts — covers the three defects below.
  3. .railway/railway.ts + .env.example — the deploy contract.
  4. Everything else is the rename set; git log -p per commit is cleaner than the squashed diff.

Review history

Two rounds of unbiased 7-agent review with fix cycles, then a cross-review against #8 that caught three more defects this branch had on its own — all with covering red-green tests.

Caught by the earlier rounds:

  • managed.ts fail-loud regression — success was logged even when gateway activation failed.
  • Empty-<Section> crashes (show_links single over-budget link; show_incident empty summary) and a monospace-fallback mis-count when a table cell contains a newline.

Caught by the cross-review against #8 (commit 7f40b1d) — all three are things the un-run live check would have surfaced:

  1. Dropped instruction on attachment turns. promptFromMessage returned contentParts ?? text, but channels-core's msgFromTurn keeps the turn's text and its attachment parts in separate fields — parts never fold in the instruction. "Chart this" + a CSV reached the model as a bare data dump. Fix ported from feat: managed (Intelligence-hosted) channel bot — Slack + Teams via @copilotkit/channels 0.2.1 #8, which hit this first on the sibling transport. A test asserted the broken shape; replaced.
  2. Unregistered interactive components. Nothing was passed to createChannel({ components }), so on the managed path — where a click arrives as a fresh delivery with no in-process handler cache — clicks can't resolve and dead-letter. That silently stranded ConfirmWrite, the HITL gate on every write. Now registers it plus IncidentCard, with a source-scan test so a future interactive card can't be forgotten.
  3. Unauthenticated HTTP surface. createCopilotNodeListener publishes the whole v2 route set — agent/:id/run, threads/list, threads/messages, memories/*, transcribe — with no auth, and this host registers agents.kite pointed at AGENT_URL: an open proxy to the brain and to thread history for anyone who can reach the port. Now gated via hooks.onRequest404 when CHANNEL_HTTP_TOKEN is unset (default), 401 on missing/wrong bearer when it is, constant-time compare. Verified against a real listener: 404/404/404 closed, 401/401/pass gated.

Testing

  • pnpm check-types: 0 errors (now includes .railway/).
  • pnpm test: 122 passing (12 files), incl. red-green tests for every behavioral fix.
  • Not done — live gateway verification (operator): set INTELLIGENCE_*, run pnpm agent + pnpm channel against opentag-dev, confirm kite-opentag flips Waiting for runtime → live, then upload a CSV with "chart this" (exercises fix 1) and click through a confirm_write gate (exercises fix 2). This PR does not deploy the channel to Railway.

Deferred — pre-existing, unrelated to this migration

Kept out to preserve reviewability; filed as follow-ups.

  • app/index.ts: partial platform creds silently skipped + misleading "No platform secrets found"; whitespace-only PORT crashes (vs managed.ts, hardened here).
  • confirm-write-tool.tsx: awaitChoice rejection unwrapped (gate stays safe; error opaque).
  • page-list.tsx: uncapped title/snippet (silent-truncation risk).
  • render-chart/render-diagram: (e as Error).message → "undefined" on non-Error throw; caption uses *x* (italic) not **x** (bold); misleading status when only the upload throws.
  • issue-card/issue-list: .slice() can split an astral-plane character.
  • .env.example: presence-gated secrets ship non-blank placeholders → a copy-paste .env treats them as configured.
  • .railway/railway.ts: builder pinning inconsistent across services.
  • vitest.config.ts: dead esbuild JSX block — confirmed live, the run prints Both esbuild and oxc options were set… esbuild options will be ignored.
  • Docs/naming: modal path bypasses confirm_write but docstring/setup imply a universal gate; NOTION_MCP_HOST undocumented; render-tools.test.ts misnamed; dead findByType helper; imprecise "≤5 inputs" title.

Worth porting from #8 afterwards

  • Child-process chart isolation (a Chromium crash can't take down the host) — but fix its two bugs first: process.exit(0) immediately after process.stdout.write(png) can truncate the PNG on a pipe, and the parent resolves that buffer with no length check; and the execFile timeout SIGTERMs the worker without a handler, so browser.close() never runs and Chromium orphans.
  • The hardened render_chart schema coercions (numeric labels, quoted numbers, tolerant options).
  • Canonical cross-platform reaction normalization (🔄 → emoji: "refresh" on every provider).
  • runner: new InMemoryAgentRunner({ onConcurrentRun: "supersede" }) in runtime.ts.
  • Teams, via createChannel({ provider: "teams" }).

Design + plan + discovery: docs/superpowers/specs/2026-07-24-opentag-intelligence-v2-migration-design.md, docs/superpowers/plans/2026-07-24-opentag-intelligence-v2-migration.md, docs/superpowers/specs/2026-07-24-opentag-intelligence-v2-migration-discovery.md.

🤖 Generated with Claude Code

jerelvelarde and others added 12 commits July 24, 2026 08:59
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…API)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nel rename set)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…te-opentag)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…managed-channels runtime

- app/index.ts: self-hosted createBot->createChannel (behavior unchanged)

- app/managed.ts: rewrite to createChannel + CopilotRuntime({channels}) + createCopilotNodeListener; drop startChannelsOverRealtimeGateway + org/project/channel-id env

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…doc/comment cleanup

Now transitive via @copilotkit/channels + @copilotkit/runtime; v2 managed.ts no longer imports it directly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
managed.ts: await channels.ready() + exit-1 on activation reject; server 'error' handler; blank INTELLIGENCE_CHANNEL_NAME normalized; extension-agnostic entry guard. IaC/docs: INTELLIGENCE_API_URL/GATEWAY_WS_URL now preserve() (drift + dev-leak); .railway added to tsconfig. commands: shared safely()/safePost() guards for bare postEphemeral/openModal/post. HITL: confirm_write detail capped; file-issue log prefix. Slack char-budget clamps: issue-list count, showcase fields/links, render-table monospace fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…essions

showcase-tools: clampWithinBudget keeps >=1 item (show_links empty-Section crash on a single over-budget link); show_incident summary .min(1) + Section guarded; MAX_LINKS budget 3000->2800 (margin parity + mrkdwn escape-expansion). render-table: sanitize embedded newlines in monospace fallback (fixes column alignment + split-based shownRows accounting). Comment accuracy: managed.ts brain framing (agent/ ships), tools/index tool-order, HITL 'later wave' marker, .railway port-pinning invariant scope.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…t/diagram rendering

The channel host runs the Playwright-based render_chart/render_diagram tools; the IaC never installed Chromium, so those failed on the one-click deploy. Add a build step (playwright install --with-deps chromium), PLAYWRIGHT_BROWSERS_PATH=0 so the browser ships in node_modules, and RAILPACK_DEPLOY_APT_PACKAGES for the runtime libs — mirrors the kite reference deploy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewing PR #8 (feat/intelligence-adapter) alongside this branch surfaced
three defects on the managed path. All three are things the un-run live
gateway check would have caught.

1. Dropped instruction on attachment turns. `promptFromMessage` returned
   `contentParts ?? text`, but channels-core's `msgFromTurn` keeps the turn's
   text and its attachment parts in SEPARATE fields — parts never fold in the
   instruction. So "chart this" + a CSV reached the model as a bare data dump
   and came back "what would you like me to do?". Lead with the instruction,
   then the parts. PR #8 hit and fixed this on the sibling transport; the fix
   is ported from there. A test asserted the broken shape — replaced.

2. Unregistered interactive components. Nothing was passed to
   `createChannel({ components })`, so on the managed path — where a click
   arrives as a fresh delivery, with no in-process handler cache — a click
   cannot be resolved and dead-letters. That silently strands `ConfirmWrite`,
   the HITL gate on every write. Register it plus `IncidentCard`, and add a
   source-scan test so a future interactive card cannot be forgotten.

3. Unauthenticated HTTP surface. `createCopilotNodeListener` publishes the
   whole v2 route set (agent/:id/run, threads/*, memories/*, transcribe) with
   no auth, and this host registers `agents.kite` pointed at AGENT_URL —
   an open proxy to the brain and to thread history for anyone who can reach
   the port. Gate it via `hooks.onRequest`: closed (404) unless
   CHANNEL_HTTP_TOKEN is set, bearer-gated (401) when it is, constant-time
   compare. Verified against a real listener: 404/404/404 closed,
   401/401/pass gated.

pnpm check-types: 0 errors. pnpm test: 122 passing (was 116).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`.migration-discovery.md` was a scratch artifact sitting at the repo root,
ungitignored, and contradicted by the PR that carried it: it records the
runtime floor staying at ^1.62.3 and channels-intelligence as a direct dep,
while the shipped branch raises the floor to ^1.63.2 and drops that dep.

Its 571 lines of verified SDK signatures are worth keeping, so move it beside
the design doc and plan under docs/superpowers/specs/ (the convention PRs #6
and #7 established) and head it with a note marking it a point-in-time record
plus the two places the implementation deliberately diverged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jerelvelarde

Copy link
Copy Markdown
Collaborator Author

Review — v2 managed-channels migration

Scope: the PR diff only (38 files, +2659/−442). I verified the SDK contracts this PR depends on against the published @copilotkit/runtime@1.63.2 and @copilotkit/channels-core@0.2.1 type definitions rather than taking the PR description at face value.

Overview

Replaces the deprecated startChannelsOverRealtimeGateway launcher with the v2 managed-channels runtime (createChannelCopilotRuntime({ channels })createCopilotNodeListener), drops the org/project/channel ID triple in favour of API-creds + channel-name derivation, and carries a mechanical Bot*Channel* rename across app/**. Config, docs, and the Railway IaC follow.

The migration itself is sound. Confirmed against the SDK types that createChannel({ components }), hooks.onRequest (documented: "Throw a Response to short-circuit"), listener.channels.ready()/stop(), and the provider default of "slack" all exist and are used correctly. No stale createBot / BotTool / channels-intelligence references survive anywhere outside the lockfile and docs.

Blocking: the success log can still lie

app/managed.ts:291-297ready() does not mean the channel is live. From ChannelsControl:

ready(): "Resolve once every declared Channel has settled to a terminal, non-connecting state (online or setup_required)."
setup_required: "the Channel is declared but has no managed provider yet — a valid degraded state, not a failure."

Confirmed in channel-manager.mjs:215 — a SETUP_REQUIRED activation error sets the status and resolves settled. So a channel that exists in the dashboard but has no Slack connector bound resolves ready() cleanly, and this code prints (channel live) and stays up. That is the most likely first-run state for a deployer, and it is the same fail-loud regression class round 1 fixed. Gate the success line on status():

await listener.channels?.ready?.({ timeoutMs: 30_000 });
const health = listener.channels?.status();
if (health && health.overall !== "online") {
  console.error(`[channel] channel not live: ${JSON.stringify(health.channels)}`);
  process.exit(1);
}

Should fix

  • No ready() timeout (app/managed.ts:291). ready({ timeoutMs }) is supported and applied per channel. Without it, an activation that never settles leaves the process bound and silent, with no success or failure line — and the Railway channel service has no healthcheck, so nothing catches it.
  • Post-boot health is unobserved. Per ChannelManager, a session that exhausts its bounded reconnect window moves to error. Nothing polls status() after boot, so KiteBot can go offline with the process healthy and restartPolicyType: ON_FAILURE never fires. Same bug as above, one lifecycle stage later.
  • buildCommand drops --frozen-lockfile (.railway/railway.ts:90). Overriding Railpack's install step with bare pnpm install means a drifted lockfile can resolve different versions in production than the ones reviewed here.
  • The Chromium install is the riskiest unverified change. --with-deps needs root apt in the build layer and every runtime lib is already enumerated in RAILPACK_DEPLOY_APT_PACKAGES — plain npx playwright install chromium is strictly less fragile. It also assumes PLAYWRIGHT_BROWSERS_PATH=0 is visible at build time; if it is not, the browser lands in ~/.cache/ms-playwright and gets stripped. Given no live deploy in this PR, this path deserves one railway up before merge.

Nits

  • The component-scan guard in app/managed.test.ts only matches ^export function (\w+)\(. export const Foo = (...) => ... — the declaration style already used for fileIssueSubmit and the tool objects in those same files — slips through, so "a future interactive card can't be forgotten" is narrower than the comment claims.
  • Registering components buys durable action resolution only as far as the ActionStore is durable. With the default in-memory store a restart still loses actions; the MANAGED_COMPONENTS docstring (app/managed.ts:56-67) reads as if registration alone is sufficient.
  • app/managed.ts:277 labels every server error as "failed to bind". A signal arriving before listen resolves yields ERR_SERVER_NOT_RUNNING from server.close(), reported as a bind failure.
  • app/managed.ts:319server.close() is not awaited before process.exit. Harmless while the surface is closed by default; worth saying so in a comment.
  • app/managed.ts:252 — one static identifyUser identity means every CHANNEL_HTTP_TOKEN bearer shares a thread/memory namespace. Correct for the stated use; worth a line in the docstring.
  • The test uses new URL(".", import.meta.url).pathname; fileURLToPath is the correct call (percent-encoding).
  • CHANNEL_HTTP_TOKEN is documented in .env.example and the IaC but missing from both env tables in setup.md.
  • The repo has no CIgh pr checks reports nothing. "122 passing, 0 type errors" is a local-only claim on a 38-file migration. A minimal pnpm check-types && pnpm test workflow would be worth more than any remaining review round here.

What's good

The HTTP auth gate is the strongest part of the PR: default-404 (not 403) is the right posture for a surface nobody should be probing, the sha256-then-timingSafeEqual compare is length-safe, and onRequest fires before routing on every request, so the closure is total. The promptFromMessage fix is correct and the replaced test is honest about what it was asserting. clampWithinBudget and toBudgetedMonospaceTable are clean, well-bounded, and report their drops instead of truncating silently. Comment density matches the surrounding code.

On the #8 overlap: that is a coordination call, not a code question — but this is the branch based on current main, and #8's Teams support and child-process chart isolation are both portable on top of it.

jerelvelarde and others added 17 commits July 27, 2026 03:55
…he closeServer test

closeServer now calls closeAllConnections() after close() so an open SSE
agent-run stream at SIGTERM can't pin shutdown until the platform SIGKILLs
the container (skipping closeBrowser + process.exit). The prior happy-path
test set `closed = true` before invoking the callback, so it would pass even
if the implementation ignored the callback entirely; it's replaced with a
test that asserts the promise stays pending until close() actually reports
done, plus a new test for the closeAllConnections call.
…und browser close

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ants

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
closeBrowser() ended with two empty catch blocks that swallowed both a
Chromium launch failure and a close() failure, making the .catch handler
in managed.ts's shutdown path unreachable dead code. Now both failure
modes propagate so the operator gets a log line instead of a silently
orphaned browser, while the "never launched" case stays a clean no-op
and repeated calls stay safe.
…ed failure modes

The scan only recognized `export function`/`export const` and attributed an
onClick to whatever export it last saw, so `export default function`,
`export async function`, and bare declarations re-exported via `export { }`
were invisible (fails open — a real interactive component reports as
all-clear), while a `defineChannelTool({...})` config or a string constant
sitting near real onClick markup got wrongly flagged for registration
(fails closed — something unregistrable is demanded).

Strips comments before scanning so a mentioned-in-prose `onClick=` never
counts, widens the declaration matcher to cover `export default function`,
`export async function`, and `export default async function`, tracks
`export { Foo }` re-exports of bare declarations, and — the core of the
fix — throws instead of silently returning `[]` when a real onClick can't
be attributed to any eligible component-shaped declaration.

`interactiveComponentNames` gains an optional `label` parameter (defaulting
to `"<source>"`) included in thrown messages.
jerelvelarde and others added 21 commits July 27, 2026 08:31
…the module import-safe

Four ways app/managed.ts violated its own "fail fast, never sit in a
half-working state" contract:

- required() used `!v`, which lets whitespace-only env values (e.g.
  AGENT_URL="   ") boot green and then fail every turn at runtime instead of
  at boot. Trim and reject blank, same as channelName/CHANNEL_HTTP_TOKEN
  already do. AGENT_URL is also now validated as a parseable URL.
- buildAgentHeaders shipped a whitespace-only AGENT_AUTH_HEADER as a real
  Authorization header on every agent request; now trimmed and treated as
  absent.
- unhealthyChannels only looked at the per-channel status map, which
  ChannelManager.status() returns empty before activation settles
  (`{overall: "connecting", channels: {}}`) and after stop()
  (`{overall: "stopped", channels: {}}`) — so the boot gate logged
  "(channel live)" for a channel that never activated. It now folds in
  `overall` when the per-channel map has nothing to say.
- process.on("unhandledRejection"/"uncaughtException") and
  `import "dotenv/config"` ran at module scope, firing on every import —
  including managed.test.ts's import for the pure helpers, installing global
  handlers into the vitest worker and loading the developer's local .env into
  the test process. Both now live behind the existing entry-point guard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tuck channels, flush diagnostics

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rors through teardown, harden its tests

onceShutdown: a synchronously-throwing run escaped the inFlight memo
(inFlight ??= run(...).catch(...) never assigns if run() throws before
returning), leaking the exception into the signal handler and letting the
next signal start a second, competing run. Wrap the call in a try/catch so a
sync throw becomes a rejected promise BEFORE .catch is attached, joining the
same memoized promise with identical microtask timing as a rejection (an
async-IIFE wrapper also catches the throw but adds an extra microtask hop,
which broke an existing timing-sensitive test). Also corrected the docstring:
later signals are silent no-ops, not "attached" observers of the first run.

server.on("error"): the non-stopping branch called fatal() directly for a
live post-bind socket error, bypassing channels.stop()/closeBrowser() the
same way the watchdog's fatal path did before an earlier pass routed it
through runShutdown. Route the post-bind case through runShutdown(...,1) too;
the pre-bind (bind failure, listening===false) case still exits directly via
fatal() since there's nothing to tear down.

Hardened three onceShutdown assertions that could not fail against the
mutants they were meant to catch: the "settled" check used one microtask
tick instead of a real macrotask boundary; the unhandled-rejection test only
asserted a synchronous function doesn't throw; and the onError guard threw
from inside an unawaited .catch, so it could never fail its own it(). Fixed
by using a real setTimeout boundary, a process.on("unhandledRejection") spy,
and an array of recorded onError calls, respectively.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ms; state the Slack provider

Three MANAGED_COMPONENTS/closeServer/activation docstrings asserted SDK/Node
behavior that installed source (@copilotkit/channels-core, @copilotkit/
channels-intelligence, @copilotkit/runtime/v2, Node's net.Server) contradicts:
component registration is insurance for a durable store, not a live dead-letter
guard; server.close()'s callback-less error was never going to reach the
'error' event; mounting the listener only constructs the channel manager,
activation is lazy on channels.ready(). Also make the implicit Slack provider
default explicit on createChannel, and rewrite the onceShutdown exitCode test
so its "onError must not fire" claim is an assertion instead of a throw that
only ever surfaces as an unrelated unhandled rejection.
…d missing env vars

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nstead of regexes

Replaces the regex-based interactive-component scanner backing the
MANAGED_COMPONENTS guard with a real ts.createSourceFile walk. A
7-agent review round independently confirmed six fails-open modes
(a type annotation, a generic arrow, memo(...), a greedy RHS peek
swallowing the next declaration, a `//` inside a string literal, and
spaced `onClick =`) and three fails-closed modes (`export default Foo;`,
an exported lowercase factory, and a template literal containing text
that looked like a declaration) that text-level parsing could not fix.

The walk climbs from each JSX `onClick` attribute to its nearest named
enclosing declaration (function or const, any RHS shape) using real AST
containment instead of textual spans, so attribution can no longer leak
across sibling declarations. A declaration is only ever reported if
exported (directly, via `export { Foo }`, or via `export default Foo;`);
among exported declarations, only capitalized names are treated as
components, which is what excludes `defineChannelTool(...)` descriptors
and lowercase factory functions without having to model what they do.
An onClick that can't be attributed to any named declaration, or that
lands in an unexported one, still throws — the fail-loud property is
preserved, not regressed.

Also deletes the "mutant proof" test block that re-implemented the old,
already-deleted regex: it could never fail in response to a change to
the shipped scanner and only diluted the suite's signal.

Verified by running the new scanner over the real app/**/*.tsx tree
(excluding __tests__/.test.), which returns exactly
["ConfirmWrite","IncidentCard"] with no throw.
…dead browsers

closeBrowser() now sets `closing` before its early return, so a shutdown
with nothing yet launched can no longer race a concurrent getBrowser()
into starting (and orphaning) a fresh Chromium. A rejected launch is
cleared from the cache instead of being remembered forever, so one
transient launch failure no longer permanently disables rendering; the
next getBrowser() retries and the failure is logged once. A `disconnected`
listener clears the cache when Chromium crashes on its own, so a dead
browser is replaced instead of failing every render forever with
Playwright's "Target closed".
…is online, validate URL scheme

Seven correctness findings in app/managed.ts:

1. server.listen's async callback had no terminal handler, so a throw
   after channels.ready() (status check, channel.name getter, setInterval,
   watchdog.unref()) became an unobserved rejection -- socket bound, no
   watchdog, no exit. Wrap it in a tracked IIFE with a .catch that routes
   to fatal(), respecting `stopping`.
2. unhandledRejection logged and kept running while uncaughtException was
   fatal, on the premise that nothing demonstrated the hazard -- Finding 1
   is that demonstration. Moved the handler inside main() (it needs
   `stopping`) and reused uncaughtExceptionAction so both handlers share
   one policy.
3. computeOverall([]) returns "online", so an empty channels map read as
   healthy -- and a test blessed that shape. unhealthyChannels now takes
   the declared channel's name and asserts it is present and online,
   rather than inferring health from an empty per-channel map.
4. requireUrl accepted anything new URL() parses, including scheme-less
   values ("agent:8123") and non-HTTP schemes. Now requires http:/https:
   and a non-empty host.
5. The watchdog's degradedSince reset on a missing health reading,
   treating "no data" as "recovered". Extracted the pure nextDegradedSince
   helper: only an affirmative overall === "online" resets the clock.
6. onceShutdown assigned its memo after invoking run, so a run that
   synchronously re-enters the returned function could start a second
   teardown. Set a placeholder guard before invoking run.
7. promptFromMessage prepended a blank text part for whitespace-only
   text ("   " is truthy). Now checked via .trim().

Findings 1, 2, and part of 5 (the setInterval callsite) live inside
main() and are not unit-testable without restructuring main(), which is
out of scope for this pass -- verified by inspection instead. Findings 3,
4, 6, 7, and the extracted nextDegradedSince helper (5) are covered by
new red-green tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…raded clock, name findings by behavior

Correct two false claims about SDK internals in app/managed.ts: the
provider:"slack" default lives in @copilotkit/runtime's
channel-activation-config.mjs (deriveChannelActivationConfig), not in
channels-core's create-channel.d.ts, which only documents the closed
union and omits the key entirely when unset. And ChannelManager.status()
after stop() does not clear entries to {} — it flips each entry's status
to "stopped" — so the post-stop shape is
{ overall: "stopped", channels: { "kite-opentag": "stopped" } }, caught
by the per-channel filter directly, not the overall fold. Fixed a test
fixture in managed.test.ts that encoded the impossible empty-map
post-stop shape.

Also: describe unhealthyChannels' actual fold-in condition (broader than
"the map is empty") and add the missing test for a degraded overall
alongside an all-online map; add the nextDegradedSince test block the
prior pass extracted the helper for but never wrote (red-green verified
the undefined-preserves-the-clock case); pass the missing filename to
interactiveComponentNames in the app-tree scan so a CI failure names a
file; and reword "Finding N" review-process labels in comments and test
names to describe the behavior they guard instead.

Comment/test-only: no executable-line changes in app/managed.ts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…gate and the watchdog

Liveness was defined three different ways: the boot gate checked the
declared channel by name, the watchdog and the degraded-clock policy
each branched on raw `overall`. They disagreed on the exact shape the
boot gate was hardened to catch — `{ overall: "online", channels: {} }`,
which ChannelManager.computeOverall genuinely returns for zero-length
input — leaving the watchdog permanently quiet and the degraded clock
reset every tick for a host whose declared channel silently dropped out
of the manager's entries.

Extract the single predicate (`isChannelLive`) and make unhealthyChannels,
channelWatchdogTick, and nextDegradedSince all consult it. Also: report
the declared channel's real condition (`name=absent (overall=...)`)
instead of the self-contradicting `overall=online`; make the by-name
presence check unconditional so it isn't buried behind an unrelated
channel's bad status; make the setup_required remediation advice
conditional on that status actually applying (new `notLiveMessage`); and
replace ChannelHealth's widened `string` fields with the SDK's closed
ChannelStatus union so a renamed status fails typecheck instead of
silently disabling the comparison.

A missing health reading now escalates after CHANNEL_DEGRADED_FATAL_MS
instead of staying quiet forever, matching nextDegradedSince's existing
"no reading != recovered" stance.
…stent

Six fail-loud gaps in app/managed.ts where an exit path could silently not
exit, or exit inconsistently with its siblings:

1. fatal() called fs.writeSync unguarded — a non-blocking pipe-backed fd 2
   throws EAGAIN under backpressure, and the throw escaped past
   process.exit entirely. Extracted guardedWrite() (swallow-and-continue,
   the one correct empty catch in this file) so the write can never
   suppress the exit that follows it.

2. The entry-point guard compared a realpath-resolved import.meta.url
   against an unresolved process.argv[1], so running via a symlink made
   the guard silently fail closed: main() never ran, process exited 0,
   zero output. Resolve argv[1] through realpathSync before comparing.

3. runShutdown was referenced (via server.on("error") and the watchdog
   interval) before its const declaration further down main() — safe only
   because nothing awaits in between today. Hoisted shutdown/runShutdown
   above every referencing call site so the TDZ hazard is structurally
   impossible, not just accidentally avoided.

4. unhandledRejection was blanket-fatal for anything not mid-shutdown, on a
   justification (the listen callback's unobserved promise) that's now
   obsolete since that IIFE has its own terminal .catch. A recurring
   library-level stray rejection (Slack SDK, Playwright, Phoenix WS,
   SanitizingHttpAgent) could burn the whole restart budget and leave the
   service stopped. New unhandledRejectionAction only escalates to fatal
   when the rejected Error's stack names this host's own module; anything
   else logs without exiting.

5. The notLive boot gate and the ready()-rejection path called fatal()
   directly even though channels.ready() already ran activate() by the
   time either fires — leaking the gateway lease instead of getting a
   clean disconnect, unlike the watchdog/server-error paths which were
   already routed through runShutdown for exactly that reason. Routed both
   (and the boot IIFE's outer catch-all, for the same reason) through
   runShutdown too.

6. shutdown()'s browser-cleanup log used plain console.error immediately
   before process.exit — the exact truncation hazard fatal() exists to
   avoid, and the only signal an operator gets that Chromium was orphaned.
   Routed it through the same guardedWrite() fatal() uses.

Findings 2, 3, and 5 are structural (entry-point ordering, TDZ hazard,
main()'s control flow) and aren't unit-testable without restructuring
main() itself — verified by inspection and full-suite regression instead.
Findings 1, 4, and 6 extract pure decisions (guardedWrite,
unhandledRejectionAction) that are directly unit tested, including RED
reproductions of each original bug.

245 -> 251 tests, 15 files, 0 type errors.
…ORT and scheme parsing

Final pass on app/managed.ts: requireUrl's dead !url.hostname check
(unreachable once the protocol is narrowed to http/https — new URL()
throws on an empty host for those schemes instead) is replaced with a
check for unresolved deployment template syntax ($, {, }, whitespace) in
the hostname, which is the actual Railway failure mode (an unresolved
${{service.VAR}} reference parses cleanly and fails every turn deep
inside SanitizingHttpAgent instead of at boot). httpAuthGate now compares
the Authorization scheme case-insensitively per RFC 7235, preserving the
constant-time credential compare. PORT parsing is extracted into a pure,
tested parsePort() that rejects hex/exponential/decimal-point input
Number() silently accepted. promptFromMessage now emits the trimmed text
instead of the raw padded value. The boot success log and channel-name
resolution in main() now thread a single normalizeChannelName() result
instead of re-deriving it from channel.name after construction.
@jerelvelarde

Copy link
Copy Markdown
Collaborator Author

Review-fix cycle complete — 38 commits pushed

Following the review above, I ran an implementation plan plus three rounds of unbiased 7-agent code review with fix cycles between them. Final state: 280 tests / 15 files, 0 type errors, actionlint clean, lockfile in sync.

The honest headline: round 1 found ~30 must-fix findings, round 2 found ~30 more — a large share created by round 1's own fixes — and round 3 found ~20. Severity dropped each round and the test suite got materially stronger (mutation testing went from catching almost nothing to 20 of 24 mutants), but no round reached zero. What follows is what landed, what didn't, and why.


The single most consequential finding

Railway's SIGTERM→SIGKILL grace period defaults to 0 seconds. app/managed.ts budgets ~10s of shutdown and four separate docstrings justified their bounds by reference to "the platform's grace period". That grace period did not exist. Every piece of shutdown hardening in this PR — bounded channels.stop(), closeAllConnections(), the browser-close deadline — was inert in the deployment it was written for.

Fixed by drainingSeconds on all three services, each derived from that service's actual teardown rather than copied: 15s channel (5s channel-stop + 5s browser race + slack), 10s notion-mcp (2× its own 5s SIGKILL escalation in scripts/start-notion-mcp.ts), 30s agent (uvicorn's graceful shutdown is unbounded when timeout_graceful_shutdown is unset, so Railway's window is the only ceiling).

My original dispatch instruction here said "neither agent nor notion-mcp has a shutdown routine to protect." That was wrong — notion-mcp has one, and it was dead code.

Fail-loud holes closed

  • The boot gate claimed liveness it hadn't verified. ChannelsControl.ready() resolves on setup_required, and ChannelManager.computeOverall([]) returns "online" for zero entries — so both "connector not bound" and "no channels at all" read as healthy. Now one isChannelLive(health, channelName) predicate is shared by the boot gate, the watchdog, and the degraded clock. Previously the gate checked by name and the watchdog checked overall, so the exact state the gate caught at boot went undetected forever afterwards.
  • ChannelHealth widened the SDK's closed status union to string, which is why those three could drift independently. Now typed to the real union.
  • The async server.listen callback had no .catch — a throw past the activation guard produced a bound socket, no watchdog, no log, no exit. Now terminal.
  • fatal() could throw instead of exiting. writeSync on a non-blocking pipe can raise EAGAIN, so the helper written to guarantee a diagnostic reached stderr could fail to die at all — and required() routes through it, so a missing env var could survive its own fail-fast.
  • The entry-point guard silently no-opped behind a symlink (import.meta.url is realpath-resolved, process.argv[1] is not) — exit code 0, no output, in a file whose whole thesis is failing loudly.
  • requireUrl guarded an impossible case and missed the real one. The empty-hostname branch was unreachable (proven across nine URL forms; a mutation harness confirmed if (false) left the suite green), while http://${{agent.RAILWAY_PRIVATE_DOMAIN}}:8123/ — the most likely Railway misconfiguration — parsed cleanly. Dead branch removed, unresolved-template check added.
  • unhandledRejection was blanket-fatal, so recurring stray rejections from any library would burn the restart budget and leave the service permanently stopped. Now scoped, with the obsolete justification replaced.
  • closeBrowser() reported success after a failed close, and its disconnected listener was attached one microtask too late. Both fixed; app/render now has test coverage it never had.
  • CHART_JS_URL / MERMAID_URL were silently dead under pnpm channel — a regression introduced by the import-safety fix in this cycle: ESM static imports evaluate before the module body, so module-scope env reads beat the deferred dotenv load. Now read lazily. mermaid also pinned to 11.16.0 (it was floating a major, executed in a --no-sandbox browser).

The interactive-component guard

The MANAGED_COMPONENTS scan failed open on export default function, export async function, deferred export { F }, adjacent export const, and a // inside a string literal — and failed closed, unsatisfiably, on tool descriptors and comment-only mentions. Two regex patches made it worse before it was rewritten on the TypeScript AST, which eliminated the class rather than the instances. Nine confirmed modes now covered as named test rows; real corpus resolves exactly ConfirmWrite and IncidentCard.

Also worth stating plainly: a reviewer traced the SDK and found registration is currently neither necessary nor sufficientActionRegistry.dispatch checks a hot cache first, clicks don't dead-letter (ActionExpiredError is swallowed), the HITL waiter resolves off evt.value independently, and across a restart store.get() throws before this.components is ever read. It stays registered as forward-looking insurance for a durable store, and the docstring now says so instead of overclaiming.

Tests that couldn't fail

Six tests in this cycle passed against the exact implementations they were written to reject. Each was rewritten and then proven by applying the named mutant and watching it fail:

  • closeServer "resolves once closed" — set its flag before invoking the callback, so a no-wait implementation passed.
  • The Promise.race replacement only caught a synchronously resolved mutant; a two-microtask-late one still passed. Replaced with a macrotask boundary.
  • closeAllConnections ordering asserted the call, not the order — swapping the two lines kept it green, while the docstring said order was the whole point.
  • Three onceShutdown tests: a memo-resets-on-settle mutant survived; an unhandled-rejection test asserted not.toThrow() on a void function; an onError guard threw into a promise nobody awaited, so it could never fail its own block.

Root cause, per the convergence audit: the assertion sat outside the channel the test runner observes. Also removed a ~90-line "mutant proof" block that tested a local copy of a regex no longer in the repo — it could never fail in response to any change to the shipped scanner. I had praised it as a durable artifact; it was history dressed as a guard.

Docs and citations

Several docstrings asserted SDK or Node behavior that was wrong, including three I wrote:

  • "Mounting the listener activates the channel" — false; activation is lazy on the first ready(). A reader could have deleted the ready() call as redundant.
  • closeServer's claim that a callback keeps ERR_SERVER_NOT_RUNNING off the 'error' event — Node drops that error entirely without a callback; the hazard couldn't occur.
  • The provider default cited channels-core's .d.ts, which disproves it; the default is applied in @copilotkit/runtime's channel-activation-config.mjs.
  • status() after stop() was documented as { channels: {} } — it isn't; entries survives. A test fixture asserted a state the runtime cannot produce.
  • The CHANNEL_HTTP_TOKEN "point a local frontend at this runtime" example can't work — no cors is passed, so a browser preflight hits the auth gate.
  • .env.example documented PORT twice with contradictory defaults, and the restart budget drifted twice (5 vs 10). The README now cites .railway/railway.ts rather than restating a number.

Also: NOTION_MCP_AUTH_TOKEN=choose-a-strong-shared-secret shipped as a working committed value — the only bearer protecting a sidecar holding a real NOTION_TOKEN. Blanked, along with the other placeholders that made cp .env.example .env start adapters with junk credentials instead of cleanly skipping them.

The Chromium apt list was derived from Playwright's own native-deps table (found bundled in playwright-core/lib/coreBundle.js) and switched to the t64 names for Ubuntu 24.04. Listing both variants was rejected on evidence: apt-get install hard-fails the entire command on an unresolvable name, and the t64 names don't exist on Jammy at all.


Deliberately not fixed

One accepted asymmetry

uncaughtException and the recognizably-ours unhandledRejection branch exit directly rather than through teardown, unlike the boot and watchdog paths — so they can leak the gateway lease. Kept deliberately: a synchronous throw or a rejection traced to this file may mean corrupted state, where attempting more async teardown is itself a risk. Flagged as a documented difference, not an oversight.

Follow-up PRs (real, load-bearing, different subject)

  1. Railway IaC typed refs. The compiled graph returns "edges": [] — all cross-service wiring is hand-written ${{svc.VAR}} strings instead of ref(). No deploy ordering (this is the Notion cold-start race the README calls unavoidable), and a service rename compiles clean and breaks at deploy.
  2. Fork safety. REPO = "CopilotKit/OpenTag" is hardcoded, so a fork following the README's Railway walkthrough deploys upstream's code. The deploy succeeds; their changes never ship.
  3. e2e rot. e2e/restart-recovery.ts imports @copilotkit/slack (never a dependency here) and two symbols that no longer exist. Invisible because tsconfig.json excludes e2e while package.json still advertises e2e:restart.
  4. CI: Python agent job. agent/tests/* exist and never run, and agent/ is the default AGENT_URL target on Railway.
  5. app/index.ts env hardening. The self-hosted path still has unfixed twins of all three env fixes (requireNonBlank, header trim, URL validation). Out of subject for this PR; the helpers are already exported.

Structural levers (deferred by scope decision)

The convergence audit grouped the findings into 8 root-cause classes and proposed additive levers — wiring-capture tests (asserting components and hooks.onRequest actually reach createChannel; today deleting either keeps every test green), starvation cases per guard, settle-behaviour tests, an infra-contract test, a docs-parity test, and an app/env.ts extraction. Deferred to keep this PR reviewable. Mutation testing (Stryker) was rejected at audit time partly because app/managed.ts couldn't be imported safely — that blocker is now gone, so it's the strongest next lever.

Subject-neutral backlog

vitest.config.ts's esbuild block is dead under Vitest 4 (proven three ways — setting jsxImportSource to a nonexistent package still passes all 70 JSX tests, because tsconfig.json is the real source). vitest.config.ts isn't typechecked. .npmrc's legacy-peer-deps is an npm setting pnpm ignores. notion-mcp has no healthcheck and no builder pin. No watchPatterns, so a README commit rebuilds and bounces all three services. No formatter or linter is configured in this repo at all — CI runs typecheck and tests only.


Not verified

No live gateway run, and no deploy. Everything above is static analysis, unit tests, and SDK/Node source reading. The operator steps from the original plan remain outstanding: set INTELLIGENCE_*, run pnpm agent + pnpm channel against opentag-dev, confirm kite-opentag flips Waiting for runtime → live, upload a CSV with "chart this", and click through a confirm_write gate. Then railway up --service channel and confirm the Chromium install lands under node_modules/ (i.e. PLAYWRIGHT_BROWSERS_PATH=0 is visible at build time) — that assumption is still untested, and the t64 apt names assume a Noble base image, which I could not verify from the repo.

The #8 overlap discussion from the description is unchanged by any of this.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant