Design/progressive disclosure

.context/DECISIONS.md

@@ -45,6 +45,20 @@ For significant decisions:
 
 -->
 
+## [2026-07-16-215955] Progressive disclosure: canonical knowledge files become bounded roots with themed rollout
+
+**Status**: Accepted
+
+**Context**: Canonical knowledge files (LEARNINGS/DECISIONS/CONVENTIONS) grow without bound as entries accumulate, and the entries stay valid, so nothing can be dropped: time-sharding plus a load-excluded 'cold' bucket was already rejected (a supersession pass found only ~1.5% cold across 162 entries; recency-gating is dangerous because old approximately equals live). At sufficient scale an agent that legitimately wants system understanding reads every decision, then every learning, and exhausts its context window. Consumption discipline (headings-first via ctx index) is necessary but NOT sufficient: an agent can always choose to read the whole file, and will when it wants completeness. Consolidation does not help either: a 2026-07-16 pass moved LEARNINGS only 98 to 88 because the corpus is dense with distinct signal, not redundancy.
+
+**Decision**: Progressive disclosure: canonical knowledge files become bounded roots with themed rollout
+
+**Rationale**: Structural boundedness makes 'read it all and die' impossible rather than merely discouraged, applying the project's own 'mechanical gates over prose' principle to context loading. Storing the generated synthesis is justified precisely BECAUSE it is expensive to recompute (an LLM pass), unlike the trivially-recomputable INDEX block whose storage was pure waste and pure drift: the cost/benefit of storing derived state inverts when recomputation is costly. Reading the bounded root alone yields compressed history PLUS verbatim recent delta, a complete current picture with no staleness gap, because staging IS the un-digested remainder by construction. The staging zone therefore serves as the watermark, so no state file is needed: a .context/state/digest.json would be a second source of truth about a fact the file already states (the INDEX-block mistake in a new coat), and its one advantage (detecting a misplaced entry) is recovered by a cheap structural invariant instead. Verified against the code: the add path anchors on the first line-start '## [' and falls back to AfterHeader, so entries always land above '## Themes' even when staging is empty, meaning ctx decision/learning add needs ZERO change; and because the root is bounded, the existing ctx agent packet needs no rewire either.
+
+**Consequence**: Entry bodies now move between files, which is the clobber risk class that index.Validate exists for, so the pass must append to the theme file, verify byte-presence, and only then remove from staging; it validates preconditions, fails loud with no auto-repair (matching the learning-add clobber fix precedent), and prefers duplication over loss on crash. New structural invariants become mechanically checkable: no line-start '## [' below '## Themes'; root gists and theme files are 1:1; every entry lives in exactly one place. CONVENTIONS needs an extra trailing '## Recent' staging heading because AppendAtEnd plus '###' prose sections would nest ambiguously inside '## Themes', and conventions edits-in-place now happen in theme files behind a link. Theme proliferation remains a slow unbounded growth vector on the root; the structure is self-similar, so an overgrown theme file can itself become a root (nesting deferred, not precluded). Scope is LEARNINGS, DECISIONS, CONVENTIONS; CONSTITUTION (small by design) and TASKS (auto-archived) are excluded. The pass is agent-driven and human-gated (agent suggests themes, human can override), triggered by suggestion only from the growth nudge, /ctx-remember, and /ctx-wrap-up, never performed inline at wrap-up because the human is leaving at that point. Spec: specs/progressive-disclosure.md.
+
+---
+
 ## [2026-07-15-000000] Live ceremony credit reuses the daily throttle marker, suppressing the day's other ceremony nudge
 
 **Status**: Accepted

.context/LEARNINGS.md

@@ -15,6 +15,46 @@ DO NOT UPDATE FOR:
 -->
 
 
+---
+
+## [2026-07-18-084406] Adding a new internal/ package must clear a fixed audit gauntlet — satisfy it up front
+
+**Context**: The internal/disclosure + internal/cli/disclosure packages tripped ~6 make audit rounds while building pd-m1/m2.
+
+**Lesson**: All mechanically enforced: doc.go per package (compliance), docstring Parameters/Returns floor on ALL funcs incl. unexported (lint-docstrings), string literals must live in config/* (magic-strings), exported API and unexported helpers in SEPARATE files (mixed-visibility), <=80-char lines, DescKey<->YAML bijection (both directions), no dead exports, and TestDocGoSubcommandDrift mis-parses a group doc.go bullet like '- internal/cli/x/cmd/y:' as subcommand claims (bulletRe backtracks on the slash).
+
+**Application**: For a new package, from the first file: one doc.go with a package comment (never a package comment in a regular file); split exported vs unexported functions across files; every literal as a config constant; full Parameters/Returns docstrings; and in a group doc.go use SPACED bullets ('- ctx x sub: ...') not slash-paths, or leave documented empty like kb/doc.go.
+
+---
+
+## [2026-07-18-084406] A /ctx-task-out measurement gate caught a pre-existing bug before it could compound
+
+**Context**: pd-m1's E4 'layout proof' epic was decomposed specifically to prove-or-kill the progressive-disclosure premise that 'ctx add needs zero change' BEFORE building the mover on it. The empty-staging case failed: ctx learning add destroyed the ## Themes section.
+
+**Lesson**: The root cause was a shipped insert.AfterHeader tail-truncation bug (content[:i]+entry, dropping content[i:]), not a design flaw. The gate surfaced a real data-loss defect in ~4 tasks instead of after building on a false premise. Decomposing a milestone's load-bearing assumption into an early falsifiable task pays for itself.
+
+**Application**: When running /ctx-task-out, make the riskiest load-bearing assumption its OWN early task with a falsifiable acceptance check; treat its failure as signal about reality (a real bug), not just about the plan. Log the gate firing in the plan's Amendments.
+
+---
+
+## [2026-07-17-081010] An insert helper that returns content[:i]+x instead of content[:i]+x+content[i:] silently drops the tail
+
+**Context**: insert.AfterHeader (the fallback beforeFirstEntry takes for a knowledge file with no ## [ entries) returned content[:insertPoint]+entry, truncating the file at the insertion point and discarding everything after it. Masked in practice because a ctx-init'd file has nothing after its comment block, so the dropped tail was empty; it bites the moment any non-entry section sits below the preamble of an as-yet-entry-less file. Same family as the LEARNINGS clobber bug index.Validate guards: silent memory loss, git-only recovery.
+
+**Lesson**: A string-splice 'insert' must reattach the tail. content[:i]+x is append-with-truncation, NOT insert. The sibling Task() splice did it right (content[:i]+x+sep+content[i:]); AfterHeader was the odd one out and nobody noticed because the package had ZERO tests. EOF-anchored fixtures mask the bug precisely because the tail is empty there.
+
+**Application**: Audit any result built from content[:i] for a matching content[i:] on the other side of the inserted text; a lone content[:i] is the smell. Test insert/splice helpers with a NON-empty tail, not just an EOF-anchored fixture. When touching a data-mutating helper with no tests, adding the test is part of the fix, not optional.
+
+---
+
+## [2026-07-17-081010] Uninitialized desc.Text() returns empty, and strings.Index(s, "") == 0 makes anchor-based inserts silently match at offset 0
+
+**Context**: Writing a layout proof for the insert package, both cases passed on the first run — for the wrong reason. A *_test.go that never calls lookup.Init() (via TestMain) gets "" back from every desc.Text() call, because the embedded asset lookup is uninitialized in that test binary. Anchor logic such as beforeFirstEntry does strings.Index(content, desc.Text(headingKey)); with a "" needle that returns 0, so 'insert before the anchor' prepends to the whole file (above the H1) and any assertion of the form 'entry appears before X' passes trivially.
+
+**Lesson**: strings.Index(s, "") == 0: an empty needle 'matches' at the start of any string. So a text-driven test whose helper resolves labels through desc.Text MUST initialize the asset lookup, or it exercises a degenerate offset-0 code path that production never takes — and passes for the wrong reason. This nearly caused a false 'measurement gate validated' report when the gate was in fact broken.
+
+**Application**: Any package test that calls desc.Text (directly, or transitively through the code under test) needs a TestMain calling lookup.Init() (see internal/cli/system/core/session/testmain_test.go for the pattern). When a text/anchor-driven test passes suspiciously easily, dump desc.Text() of the keys involved and assert they are non-empty before trusting the assertions.
+
 ---
 
 ## [2026-07-16-120001] OpenCode plugin integration gotchas (consolidated)

.context/TASKS.md

@@ -2933,13 +2933,66 @@ sweep completes. No new flags.
 
 ### Future
 
+- [ ] PD-M3: the mover — append->verify->remove + gist write-back; first milestone that WRITES canonical files (clobber risk class). Decompose via /ctx-task-out --milestone pd-m3; consumes the disclosure.Inspection built in M2. #session:87e465a0 #branch:design/progressive-disclosure #commit:2ff82775 #added:2026-07-18-084419
+
 - [ ] Hub curation: immutable promotion ledger (who accepted what, when, why) + mechanical validate floor for shared knowledge; revisit when ctx hub grows team-curation workflows #priority:low #session:a31b3e67 #branch:main #commit:d800734c #added:2026-07-04-153004
 
 - [ ] ctx-spec-views skill: manager-facing read-models (execution plan, spec briefs, task breakdowns) generated FROM specs/, never source of truth; spec it when someone actually needs the leadership view #priority:low #session:a31b3e67 #branch:main #commit:d800734c #added:2026-07-04-153004
 
 - [ ] KB convention: pinned upstream corpus for grounding — document a snapshot mode (dated local copy of high-churn upstream docs as the citable byte-stream) in KB rules; no code needed #priority:low #session:a31b3e67 #branch:main #commit:d800734c #added:2026-07-04-153004
 
+### Phase PD-M1: Progressive Disclosure — Guards, Invariants, Vocabulary
+
+Plan: `specs/plans/pd-m1.md` · Spec: `specs/progressive-disclosure.md`
+Milestone 1 builds the refusal machinery and proves the layout premise.
+**Nothing moves** — no entry body is relocated and no gist is authored;
+the pass (M2+) that moves bodies is the clobber risk class, so guards
+land first.
+
+**Completion rule**: an epic below is checked `[x]` only when every task
+in its range is `[x]` or `[o]` in `specs/plans/pd-m1.md`. The plan — not
+this list — is the single source of truth for milestone progress.
+
+- [x] [E1] Structural vocabulary, types, error sentinels (T01–T03). Plan: specs/plans/pd-m1.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16
+
+- [x] [E2] Root parser + Validate precondition (T04–T06). Plan: specs/plans/pd-m1.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16
+
+- [x] [E3] Cross-file invariants: pairing, uniqueness, links (T07–T09). Plan: specs/plans/pd-m1.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16
+
+- [x] [E4] Layout proofs — the add-path de-risking; MEASUREMENT GATE: if these fail, the "zero change to add" premise is wrong and the spec's Layout section must be revisited via /ctx-plan (T10–T12). Plan: specs/plans/pd-m1.md #priority:high #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16
+  DONE 2026-07-17: gate FIRED (T10 empty-staging destroyed ## Themes), root cause was the pre-existing insert.AfterHeader tail-truncation bug — fixed separately (specs/fix-afterheader-tail-truncation.md, merged), T10–T12 now green unchanged. Design premise holds.
+
+- [x] [E5] doc.go, compliance wiring, milestone gate (T13–T15). Plan: specs/plans/pd-m1.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16
+
+### Phase PD-M2: Progressive Disclosure — the dry-run pass (inspect + propose)
+
+Plan: `specs/plans/pd-m2.md` · Spec: `specs/progressive-disclosure.md`
+Milestone 2 delivers the digesting pass in **dry-run only**: a read-only
+`ctx disclosure inspect` CLI and a skill that proposes themes + gists and
+shows the plan — **moving nothing**. The mover (append→verify→remove) is
+M3, behind this fully-exercised read+plan path.
+
+**Completion rule**: an epic is `[x]` only when every task in its range is
+`[x]`/`[o]` in `specs/plans/pd-m2.md` (the plan is the source of truth).
+
+- [x] [E1] Disclosure inspect model: KindFor, StagedEntries, Inspect (T01–T04). Plan: specs/plans/pd-m2.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-18
+
+- [x] [E2] `ctx disclosure inspect` CLI — read-only, JSON, write-nothing, non-knowledge-file rejection (T05–T10). Plan: specs/plans/pd-m2.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-18
+
+- [x] [E3] `ctx-digest` dry-run skill + copilot sync + milestone gate; MEASUREMENT GATE T12: dry-run must produce a coherent theme plan and move nothing (T11–T14). Plan: specs/plans/pd-m2.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-18
+
 - [ ] Progressive disclosure for canonical context files: the growth warnings (LEARNINGS/DECISIONS/CONVENTIONS over threshold) are NOT redundancy — consolidation only got LEARNINGS 98→88 because the entries are distinct, dense signal. The real lever is a structural pass: canonical files carry a tight summary/index and detail loads on demand (via `ctx index`/`ctx search` projection + an archive/detail tier). Manual design exercise first (/ctx-brainstorm → spec), then codify the repeatable procedure as a new skill (e.g. /ctx-progressive-disclosure). This exercise IS the baseline for the skill. #priority:medium #session:87e465a0 #branch:main #added:2026-07-16
+  DESIGN DONE 2026-07-16 (session 87e465a0): /ctx-brainstorm run to
+  completion (Understanding Lock → approaches → stress-test → design).
+  Spec: specs/progressive-disclosure.md. Decision: DECISIONS.md
+  [2026-07-16-215955]. Shape: each canonical root becomes BOUNDED —
+  staging zone (where `add` already writes, zero code change) + `## Themes`
+  gists+links; bodies roll out to .context/<noun>/<theme>.md; staging IS
+  the watermark (no state file); structure self-similar so nesting is
+  available but deferred. Scope LEARNINGS/DECISIONS/CONVENTIONS; excludes
+  CONSTITUTION + TASKS. IMPLEMENTATION STILL OPEN — phasing sketched in
+  the spec (guards+invariants first, then the pass-as-skill dry-run, then
+  rollout). The skill itself is the final deliverable.
 
 - [ ] Journal parser schema drift: `ctx journal import` reports "Schema drift detected" — Claude Code transcripts now emit fields/records the parser doesn't recognize: unknown fields classifierMetaLines, session_id, toolDenialKind; unknown record type file-history-delta; unknown block type fallback. Import still works but silently ignores them. Update internal/journal parser to recognize (or explicitly skip) the new CC schema; run `ctx journal schema check` for the full report. #priority:medium #session:87e465a0 #branch:main #added:2026-07-16
 

internal/assets/claude/skills/ctx-digest/SKILL.md

@@ -0,0 +1,91 @@
+---
+name: ctx-digest
+description: "Dry-run the progressive-disclosure pass: inspect a knowledge file's staging zone, propose a theme per staged entry, author gists, and show the digest plan — WITHOUT moving anything. Use to preview how LEARNINGS.md / DECISIONS.md would fold into themes when a file grows large."
+allowed-tools: Bash(ctx:*), Read
+---
+
+Preview how a bounded knowledge root would digest its staging zone into
+themes — and **move nothing**. This is the dry-run half of progressive
+disclosure (see specs/progressive-disclosure.md): it proposes and shows
+a plan; a later milestone's apply step does the moving.
+
+**This skill never edits a knowledge file, never creates a theme file,
+and never runs an apply/move.** Its entire output is a proposed plan for
+a human to review.
+
+## When to Use
+
+- A knowledge file (LEARNINGS.md, DECISIONS.md) has grown large and you
+  want to see how it *would* be folded into themes
+- The knowledge-growth nudge fired and you want a concrete digest plan
+  before committing to restructuring
+- To sanity-check theme groupings before the apply step exists
+
+## When NOT to Use
+
+- To actually move entries — the mover is a later milestone; this skill
+  is preview-only
+- On CONSTITUTION.md or TASKS.md — out of scope (small by design /
+  auto-archived)
+- On CONVENTIONS.md — its entries are `##` sections, not `## [` entries;
+  staged-entry digestion there is a separate milestone
+
+## Procedure
+
+### 1. Inspect the root
+
+```bash
+ctx disclosure inspect .context/LEARNINGS.md --json
+```
+
+This reports, as JSON: the `kind`, the `staging` entries (timestamp +
+title, un-digested), and the current `themes` (name, gist, link). Read
+it; do not parse the file by hand.
+
+If `staging` is empty, there is nothing to digest — say so and stop.
+
+### 2. Propose a theme per staged entry
+
+For each staged entry, assign it to a **theme** — an existing theme (by
+name) or a new one. This is a semantic judgement: group entries that
+share a subject (e.g. "hook mechanics", "error handling", "OpenCode
+integration"), not by date.
+
+- Keep themes **coarse**: a handful covering many entries beats one
+  theme per entry.
+- Prefer an **existing** theme when an entry fits it.
+- Surface your proposed grouping to the human and let them **rename,
+  merge, split, or reassign** before anything is final. Themes are the
+  human's call; you propose.
+
+### 3. Author a gist per touched theme
+
+For each theme in the plan, write the **gist** — one line, soft ceiling
+~140 chars, saying what the theme *covers* (the shape of its knowledge),
+not listing its entries. "hook mechanics: output channels, key names,
+compliance wiring" — not "entry A; entry B". The gist tells a future
+reader *whether to drill in*, nothing more. (Spec: `### Gist format`.)
+
+### 4. Present the plan — then stop
+
+Show the human, per theme:
+
+```
+Theme: <name>   →  .context/<noun>/<slug>.md   (would create/append)
+  gist: <proposed one-line gist>
+  entries (N):
+    - [<ts>] <title>
+    - …
+```
+
+End with an explicit note: **"Dry run — nothing was moved or written.
+The apply step (a later milestone) performs the move."** Do not edit any
+file. Do not run an apply command (there is none yet).
+
+## Quality Checklist
+
+- [ ] Ran `ctx disclosure inspect --json`; did not hand-parse the file
+- [ ] Every staged entry is assigned to exactly one theme in the plan
+- [ ] Each theme has a one-line gist describing coverage (not a list)
+- [ ] The plan was surfaced for the human to adjust themes
+- [ ] **Nothing was edited, created, or moved** — output is a plan only