diff --git a/.context/DECISIONS.md b/.context/DECISIONS.md index d8602c66f..87dd2cfb9 100644 --- a/.context/DECISIONS.md +++ b/.context/DECISIONS.md @@ -45,6 +45,20 @@ For significant decisions: --> +## [2026-07-16-215955] Progressive disclosure: canonical knowledge files become bounded roots with themed rollout + +**Status**: Accepted + +**Context**: Canonical knowledge files (LEARNINGS/DECISIONS/CONVENTIONS) grow without bound as entries accumulate, and the entries stay valid, so nothing can be dropped: time-sharding plus a load-excluded 'cold' bucket was already rejected (a supersession pass found only ~1.5% cold across 162 entries; recency-gating is dangerous because old approximately equals live). At sufficient scale an agent that legitimately wants system understanding reads every decision, then every learning, and exhausts its context window. Consumption discipline (headings-first via ctx index) is necessary but NOT sufficient: an agent can always choose to read the whole file, and will when it wants completeness. Consolidation does not help either: a 2026-07-16 pass moved LEARNINGS only 98 to 88 because the corpus is dense with distinct signal, not redundancy. + +**Decision**: Progressive disclosure: canonical knowledge files become bounded roots with themed rollout + +**Rationale**: Structural boundedness makes 'read it all and die' impossible rather than merely discouraged, applying the project's own 'mechanical gates over prose' principle to context loading. Storing the generated synthesis is justified precisely BECAUSE it is expensive to recompute (an LLM pass), unlike the trivially-recomputable INDEX block whose storage was pure waste and pure drift: the cost/benefit of storing derived state inverts when recomputation is costly. Reading the bounded root alone yields compressed history PLUS verbatim recent delta, a complete current picture with no staleness gap, because staging IS the un-digested remainder by construction. The staging zone therefore serves as the watermark, so no state file is needed: a .context/state/digest.json would be a second source of truth about a fact the file already states (the INDEX-block mistake in a new coat), and its one advantage (detecting a misplaced entry) is recovered by a cheap structural invariant instead. Verified against the code: the add path anchors on the first line-start '## [' and falls back to AfterHeader, so entries always land above '## Themes' even when staging is empty, meaning ctx decision/learning add needs ZERO change; and because the root is bounded, the existing ctx agent packet needs no rewire either. + +**Consequence**: Entry bodies now move between files, which is the clobber risk class that index.Validate exists for, so the pass must append to the theme file, verify byte-presence, and only then remove from staging; it validates preconditions, fails loud with no auto-repair (matching the learning-add clobber fix precedent), and prefers duplication over loss on crash. New structural invariants become mechanically checkable: no line-start '## [' below '## Themes'; root gists and theme files are 1:1; every entry lives in exactly one place. CONVENTIONS needs an extra trailing '## Recent' staging heading because AppendAtEnd plus '###' prose sections would nest ambiguously inside '## Themes', and conventions edits-in-place now happen in theme files behind a link. Theme proliferation remains a slow unbounded growth vector on the root; the structure is self-similar, so an overgrown theme file can itself become a root (nesting deferred, not precluded). Scope is LEARNINGS, DECISIONS, CONVENTIONS; CONSTITUTION (small by design) and TASKS (auto-archived) are excluded. The pass is agent-driven and human-gated (agent suggests themes, human can override), triggered by suggestion only from the growth nudge, /ctx-remember, and /ctx-wrap-up, never performed inline at wrap-up because the human is leaving at that point. Spec: specs/progressive-disclosure.md. + +--- + ## [2026-07-15-000000] Live ceremony credit reuses the daily throttle marker, suppressing the day's other ceremony nudge **Status**: Accepted diff --git a/.context/LEARNINGS.md b/.context/LEARNINGS.md index f142d86df..2ce298456 100644 --- a/.context/LEARNINGS.md +++ b/.context/LEARNINGS.md @@ -15,6 +15,46 @@ DO NOT UPDATE FOR: --> +--- + +## [2026-07-18-084406] Adding a new internal/ package must clear a fixed audit gauntlet — satisfy it up front + +**Context**: The internal/disclosure + internal/cli/disclosure packages tripped ~6 make audit rounds while building pd-m1/m2. + +**Lesson**: All mechanically enforced: doc.go per package (compliance), docstring Parameters/Returns floor on ALL funcs incl. unexported (lint-docstrings), string literals must live in config/* (magic-strings), exported API and unexported helpers in SEPARATE files (mixed-visibility), <=80-char lines, DescKey<->YAML bijection (both directions), no dead exports, and TestDocGoSubcommandDrift mis-parses a group doc.go bullet like '- internal/cli/x/cmd/y:' as subcommand claims (bulletRe backtracks on the slash). + +**Application**: For a new package, from the first file: one doc.go with a package comment (never a package comment in a regular file); split exported vs unexported functions across files; every literal as a config constant; full Parameters/Returns docstrings; and in a group doc.go use SPACED bullets ('- ctx x sub: ...') not slash-paths, or leave documented empty like kb/doc.go. + +--- + +## [2026-07-18-084406] A /ctx-task-out measurement gate caught a pre-existing bug before it could compound + +**Context**: pd-m1's E4 'layout proof' epic was decomposed specifically to prove-or-kill the progressive-disclosure premise that 'ctx add needs zero change' BEFORE building the mover on it. The empty-staging case failed: ctx learning add destroyed the ## Themes section. + +**Lesson**: The root cause was a shipped insert.AfterHeader tail-truncation bug (content[:i]+entry, dropping content[i:]), not a design flaw. The gate surfaced a real data-loss defect in ~4 tasks instead of after building on a false premise. Decomposing a milestone's load-bearing assumption into an early falsifiable task pays for itself. + +**Application**: When running /ctx-task-out, make the riskiest load-bearing assumption its OWN early task with a falsifiable acceptance check; treat its failure as signal about reality (a real bug), not just about the plan. Log the gate firing in the plan's Amendments. + +--- + +## [2026-07-17-081010] An insert helper that returns content[:i]+x instead of content[:i]+x+content[i:] silently drops the tail + +**Context**: insert.AfterHeader (the fallback beforeFirstEntry takes for a knowledge file with no ## [ entries) returned content[:insertPoint]+entry, truncating the file at the insertion point and discarding everything after it. Masked in practice because a ctx-init'd file has nothing after its comment block, so the dropped tail was empty; it bites the moment any non-entry section sits below the preamble of an as-yet-entry-less file. Same family as the LEARNINGS clobber bug index.Validate guards: silent memory loss, git-only recovery. + +**Lesson**: A string-splice 'insert' must reattach the tail. content[:i]+x is append-with-truncation, NOT insert. The sibling Task() splice did it right (content[:i]+x+sep+content[i:]); AfterHeader was the odd one out and nobody noticed because the package had ZERO tests. EOF-anchored fixtures mask the bug precisely because the tail is empty there. + +**Application**: Audit any result built from content[:i] for a matching content[i:] on the other side of the inserted text; a lone content[:i] is the smell. Test insert/splice helpers with a NON-empty tail, not just an EOF-anchored fixture. When touching a data-mutating helper with no tests, adding the test is part of the fix, not optional. + +--- + +## [2026-07-17-081010] Uninitialized desc.Text() returns empty, and strings.Index(s, "") == 0 makes anchor-based inserts silently match at offset 0 + +**Context**: Writing a layout proof for the insert package, both cases passed on the first run — for the wrong reason. A *_test.go that never calls lookup.Init() (via TestMain) gets "" back from every desc.Text() call, because the embedded asset lookup is uninitialized in that test binary. Anchor logic such as beforeFirstEntry does strings.Index(content, desc.Text(headingKey)); with a "" needle that returns 0, so 'insert before the anchor' prepends to the whole file (above the H1) and any assertion of the form 'entry appears before X' passes trivially. + +**Lesson**: strings.Index(s, "") == 0: an empty needle 'matches' at the start of any string. So a text-driven test whose helper resolves labels through desc.Text MUST initialize the asset lookup, or it exercises a degenerate offset-0 code path that production never takes — and passes for the wrong reason. This nearly caused a false 'measurement gate validated' report when the gate was in fact broken. + +**Application**: Any package test that calls desc.Text (directly, or transitively through the code under test) needs a TestMain calling lookup.Init() (see internal/cli/system/core/session/testmain_test.go for the pattern). When a text/anchor-driven test passes suspiciously easily, dump desc.Text() of the keys involved and assert they are non-empty before trusting the assertions. + --- ## [2026-07-16-120001] OpenCode plugin integration gotchas (consolidated) diff --git a/.context/TASKS.md b/.context/TASKS.md index 36b6890df..caa0325b3 100644 --- a/.context/TASKS.md +++ b/.context/TASKS.md @@ -2933,13 +2933,66 @@ sweep completes. No new flags. ### Future +- [ ] PD-M3: the mover — append->verify->remove + gist write-back; first milestone that WRITES canonical files (clobber risk class). Decompose via /ctx-task-out --milestone pd-m3; consumes the disclosure.Inspection built in M2. #session:87e465a0 #branch:design/progressive-disclosure #commit:2ff82775 #added:2026-07-18-084419 + - [ ] Hub curation: immutable promotion ledger (who accepted what, when, why) + mechanical validate floor for shared knowledge; revisit when ctx hub grows team-curation workflows #priority:low #session:a31b3e67 #branch:main #commit:d800734c #added:2026-07-04-153004 - [ ] ctx-spec-views skill: manager-facing read-models (execution plan, spec briefs, task breakdowns) generated FROM specs/, never source of truth; spec it when someone actually needs the leadership view #priority:low #session:a31b3e67 #branch:main #commit:d800734c #added:2026-07-04-153004 - [ ] KB convention: pinned upstream corpus for grounding — document a snapshot mode (dated local copy of high-churn upstream docs as the citable byte-stream) in KB rules; no code needed #priority:low #session:a31b3e67 #branch:main #commit:d800734c #added:2026-07-04-153004 +### Phase PD-M1: Progressive Disclosure — Guards, Invariants, Vocabulary + +Plan: `specs/plans/pd-m1.md` · Spec: `specs/progressive-disclosure.md` +Milestone 1 builds the refusal machinery and proves the layout premise. +**Nothing moves** — no entry body is relocated and no gist is authored; +the pass (M2+) that moves bodies is the clobber risk class, so guards +land first. + +**Completion rule**: an epic below is checked `[x]` only when every task +in its range is `[x]` or `[o]` in `specs/plans/pd-m1.md`. The plan — not +this list — is the single source of truth for milestone progress. + +- [x] [E1] Structural vocabulary, types, error sentinels (T01–T03). Plan: specs/plans/pd-m1.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16 + +- [x] [E2] Root parser + Validate precondition (T04–T06). Plan: specs/plans/pd-m1.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16 + +- [x] [E3] Cross-file invariants: pairing, uniqueness, links (T07–T09). Plan: specs/plans/pd-m1.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16 + +- [x] [E4] Layout proofs — the add-path de-risking; MEASUREMENT GATE: if these fail, the "zero change to add" premise is wrong and the spec's Layout section must be revisited via /ctx-plan (T10–T12). Plan: specs/plans/pd-m1.md #priority:high #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16 + DONE 2026-07-17: gate FIRED (T10 empty-staging destroyed ## Themes), root cause was the pre-existing insert.AfterHeader tail-truncation bug — fixed separately (specs/fix-afterheader-tail-truncation.md, merged), T10–T12 now green unchanged. Design premise holds. + +- [x] [E5] doc.go, compliance wiring, milestone gate (T13–T15). Plan: specs/plans/pd-m1.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-16 + +### Phase PD-M2: Progressive Disclosure — the dry-run pass (inspect + propose) + +Plan: `specs/plans/pd-m2.md` · Spec: `specs/progressive-disclosure.md` +Milestone 2 delivers the digesting pass in **dry-run only**: a read-only +`ctx disclosure inspect` CLI and a skill that proposes themes + gists and +shows the plan — **moving nothing**. The mover (append→verify→remove) is +M3, behind this fully-exercised read+plan path. + +**Completion rule**: an epic is `[x]` only when every task in its range is +`[x]`/`[o]` in `specs/plans/pd-m2.md` (the plan is the source of truth). + +- [x] [E1] Disclosure inspect model: KindFor, StagedEntries, Inspect (T01–T04). Plan: specs/plans/pd-m2.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-18 + +- [x] [E2] `ctx disclosure inspect` CLI — read-only, JSON, write-nothing, non-knowledge-file rejection (T05–T10). Plan: specs/plans/pd-m2.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-18 + +- [x] [E3] `ctx-digest` dry-run skill + copilot sync + milestone gate; MEASUREMENT GATE T12: dry-run must produce a coherent theme plan and move nothing (T11–T14). Plan: specs/plans/pd-m2.md #priority:medium #session:87e465a0 #branch:design/progressive-disclosure #added:2026-07-18 + - [ ] Progressive disclosure for canonical context files: the growth warnings (LEARNINGS/DECISIONS/CONVENTIONS over threshold) are NOT redundancy — consolidation only got LEARNINGS 98→88 because the entries are distinct, dense signal. The real lever is a structural pass: canonical files carry a tight summary/index and detail loads on demand (via `ctx index`/`ctx search` projection + an archive/detail tier). Manual design exercise first (/ctx-brainstorm → spec), then codify the repeatable procedure as a new skill (e.g. /ctx-progressive-disclosure). This exercise IS the baseline for the skill. #priority:medium #session:87e465a0 #branch:main #added:2026-07-16 + DESIGN DONE 2026-07-16 (session 87e465a0): /ctx-brainstorm run to + completion (Understanding Lock → approaches → stress-test → design). + Spec: specs/progressive-disclosure.md. Decision: DECISIONS.md + [2026-07-16-215955]. Shape: each canonical root becomes BOUNDED — + staging zone (where `add` already writes, zero code change) + `## Themes` + gists+links; bodies roll out to .context//.md; staging IS + the watermark (no state file); structure self-similar so nesting is + available but deferred. Scope LEARNINGS/DECISIONS/CONVENTIONS; excludes + CONSTITUTION + TASKS. IMPLEMENTATION STILL OPEN — phasing sketched in + the spec (guards+invariants first, then the pass-as-skill dry-run, then + rollout). The skill itself is the final deliverable. - [ ] Journal parser schema drift: `ctx journal import` reports "Schema drift detected" — Claude Code transcripts now emit fields/records the parser doesn't recognize: unknown fields classifierMetaLines, session_id, toolDenialKind; unknown record type file-history-delta; unknown block type fallback. Import still works but silently ignores them. Update internal/journal parser to recognize (or explicitly skip) the new CC schema; run `ctx journal schema check` for the full report. #priority:medium #session:87e465a0 #branch:main #added:2026-07-16 diff --git a/internal/assets/claude/skills/ctx-digest/SKILL.md b/internal/assets/claude/skills/ctx-digest/SKILL.md new file mode 100644 index 000000000..993fdf59e --- /dev/null +++ b/internal/assets/claude/skills/ctx-digest/SKILL.md @@ -0,0 +1,91 @@ +--- +name: ctx-digest +description: "Dry-run the progressive-disclosure pass: inspect a knowledge file's staging zone, propose a theme per staged entry, author gists, and show the digest plan — WITHOUT moving anything. Use to preview how LEARNINGS.md / DECISIONS.md would fold into themes when a file grows large." +allowed-tools: Bash(ctx:*), Read +--- + +Preview how a bounded knowledge root would digest its staging zone into +themes — and **move nothing**. This is the dry-run half of progressive +disclosure (see specs/progressive-disclosure.md): it proposes and shows +a plan; a later milestone's apply step does the moving. + +**This skill never edits a knowledge file, never creates a theme file, +and never runs an apply/move.** Its entire output is a proposed plan for +a human to review. + +## When to Use + +- A knowledge file (LEARNINGS.md, DECISIONS.md) has grown large and you + want to see how it *would* be folded into themes +- The knowledge-growth nudge fired and you want a concrete digest plan + before committing to restructuring +- To sanity-check theme groupings before the apply step exists + +## When NOT to Use + +- To actually move entries — the mover is a later milestone; this skill + is preview-only +- On CONSTITUTION.md or TASKS.md — out of scope (small by design / + auto-archived) +- On CONVENTIONS.md — its entries are `##` sections, not `## [` entries; + staged-entry digestion there is a separate milestone + +## Procedure + +### 1. Inspect the root + +```bash +ctx disclosure inspect .context/LEARNINGS.md --json +``` + +This reports, as JSON: the `kind`, the `staging` entries (timestamp + +title, un-digested), and the current `themes` (name, gist, link). Read +it; do not parse the file by hand. + +If `staging` is empty, there is nothing to digest — say so and stop. + +### 2. Propose a theme per staged entry + +For each staged entry, assign it to a **theme** — an existing theme (by +name) or a new one. This is a semantic judgement: group entries that +share a subject (e.g. "hook mechanics", "error handling", "OpenCode +integration"), not by date. + +- Keep themes **coarse**: a handful covering many entries beats one + theme per entry. +- Prefer an **existing** theme when an entry fits it. +- Surface your proposed grouping to the human and let them **rename, + merge, split, or reassign** before anything is final. Themes are the + human's call; you propose. + +### 3. Author a gist per touched theme + +For each theme in the plan, write the **gist** — one line, soft ceiling +~140 chars, saying what the theme *covers* (the shape of its knowledge), +not listing its entries. "hook mechanics: output channels, key names, +compliance wiring" — not "entry A; entry B". The gist tells a future +reader *whether to drill in*, nothing more. (Spec: `### Gist format`.) + +### 4. Present the plan — then stop + +Show the human, per theme: + +``` +Theme: → .context//.md (would create/append) + gist: + entries (N): + - [] + - … +``` + +End with an explicit note: **"Dry run — nothing was moved or written. +The apply step (a later milestone) performs the move."** Do not edit any +file. Do not run an apply command (there is none yet). + +## Quality Checklist + +- [ ] Ran `ctx disclosure inspect --json`; did not hand-parse the file +- [ ] Every staged entry is assigned to exactly one theme in the plan +- [ ] Each theme has a one-line gist describing coverage (not a list) +- [ ] The plan was surfaced for the human to adjust themes +- [ ] **Nothing was edited, created, or moved** — output is a plan only diff --git a/internal/assets/commands/commands.yaml b/internal/assets/commands/commands.yaml index 60a48fe9d..56b93c479 100644 --- a/internal/assets/commands/commands.yaml +++ b/internal/assets/commands/commands.yaml @@ -632,6 +632,28 @@ index: ctx index .context/TASKS.md --depth 3 ctx index .context/LEARNINGS.md --json short: Project a file's headings as a table of contents +disclosure: + long: |- + Inspect the canonical knowledge files under progressive disclosure. + + A knowledge file's staging zone holds recent, un-digested entries; a + ## Themes section carries a gist and a link per theme. ctx disclosure + reports that structure so the digesting pass can propose how to fold + staged entries into their theme files. + short: Inspect knowledge files under progressive disclosure +disclosure-inspect: + long: |- + Report a knowledge file's staging entries and current themes. + + ctx disclosure inspect reads a canonical knowledge file (LEARNINGS.md, + DECISIONS.md, or CONVENTIONS.md) and prints which entries are staged + (awaiting digestion) and which themes already exist. It is read-only + and never writes. Use --json for machine-readable output. + + Examples: + ctx disclosure inspect .context/LEARNINGS.md + ctx disclosure inspect .context/DECISIONS.md --json + short: Report a knowledge file's staged entries and themes load: long: |- Load and display the assembled context diff --git a/internal/assets/commands/text/errors.yaml b/internal/assets/commands/text/errors.yaml index d865ff629..7e4841f96 100644 --- a/internal/assets/commands/text/errors.yaml +++ b/internal/assets/commands/text/errors.yaml @@ -140,6 +140,24 @@ err.dream.write-scope: short: 'ctx-dream guard: write outside dream scope refused: %s' err.dream.write-state: short: 'dream: write state %s: %w' +err.disclosure.broken-theme-link: + short: 'progressive disclosure: a theme link in the root points at a path that does not exist' +err.disclosure.duplicate-entry: + short: 'progressive disclosure: an entry appears in more than one place (staging and/or multiple theme files)' +err.disclosure.entry-below-themes: + short: 'progressive disclosure: a "## [" entry sits below the "## Themes" section; entries must stay in the staging zone above it' +err.disclosure.missing-theme-file: + short: 'progressive disclosure: a theme gist in the root has no matching theme file' +err.disclosure.multiple-themes: + short: 'progressive disclosure: the root has more than one "## Themes" section' +err.disclosure.not-knowledge-file-msg: + short: 'not a canonical knowledge file' +err.disclosure.not-knowledge-file: + short: '%w: %s (expected LEARNINGS.md, DECISIONS.md, or CONVENTIONS.md)' +err.disclosure.orphan-theme-file: + short: 'progressive disclosure: a theme file has no matching gist in the root' +err.disclosure.staging-unparsable: + short: 'progressive disclosure: the staging zone could not be parsed into discrete entries' err.crypto.ciphertext-too-short: short: ciphertext too short err.crypto.create-cipher: diff --git a/internal/assets/commands/text/write.yaml b/internal/assets/commands/text/write.yaml index 8fddf3947..42ef7f99a 100644 --- a/internal/assets/commands/text/write.yaml +++ b/internal/assets/commands/text/write.yaml @@ -750,6 +750,18 @@ write.status-file-compact: short: ' %s %s (%s)' write.status-file-verbose: short: ' %s %s (%s) [%s tokens, %s]' +write.disclosure-kind: + short: 'Kind: %s' +write.disclosure-staged-header: + short: 'Staged (%d):' +write.disclosure-staged-line: + short: ' [%s] %s' +write.disclosure-themes-header: + short: 'Themes (%d):' +write.disclosure-theme-line: + short: ' %s → %s' +write.disclosure-none: + short: ' (none)' write.status-header-block: short: |- Context Status diff --git a/internal/assets/integrations/copilot-cli/skills/ctx-digest/SKILL.md b/internal/assets/integrations/copilot-cli/skills/ctx-digest/SKILL.md new file mode 100644 index 000000000..c4a7cb24e --- /dev/null +++ b/internal/assets/integrations/copilot-cli/skills/ctx-digest/SKILL.md @@ -0,0 +1,90 @@ +--- +name: ctx-digest +description: "Dry-run the progressive-disclosure pass: inspect a knowledge file's staging zone, propose a theme per staged entry, author gists, and show the digest plan — WITHOUT moving anything. Use to preview how LEARNINGS.md / DECISIONS.md would fold into themes when a file grows large." +--- + +Preview how a bounded knowledge root would digest its staging zone into +themes — and **move nothing**. This is the dry-run half of progressive +disclosure (see specs/progressive-disclosure.md): it proposes and shows +a plan; a later milestone's apply step does the moving. + +**This skill never edits a knowledge file, never creates a theme file, +and never runs an apply/move.** Its entire output is a proposed plan for +a human to review. + +## When to Use + +- A knowledge file (LEARNINGS.md, DECISIONS.md) has grown large and you + want to see how it *would* be folded into themes +- The knowledge-growth nudge fired and you want a concrete digest plan + before committing to restructuring +- To sanity-check theme groupings before the apply step exists + +## When NOT to Use + +- To actually move entries — the mover is a later milestone; this skill + is preview-only +- On CONSTITUTION.md or TASKS.md — out of scope (small by design / + auto-archived) +- On CONVENTIONS.md — its entries are `##` sections, not `## [` entries; + staged-entry digestion there is a separate milestone + +## Procedure + +### 1. Inspect the root + +```bash +ctx disclosure inspect .context/LEARNINGS.md --json +``` + +This reports, as JSON: the `kind`, the `staging` entries (timestamp + +title, un-digested), and the current `themes` (name, gist, link). Read +it; do not parse the file by hand. + +If `staging` is empty, there is nothing to digest — say so and stop. + +### 2. Propose a theme per staged entry + +For each staged entry, assign it to a **theme** — an existing theme (by +name) or a new one. This is a semantic judgement: group entries that +share a subject (e.g. "hook mechanics", "error handling", "OpenCode +integration"), not by date. + +- Keep themes **coarse**: a handful covering many entries beats one + theme per entry. +- Prefer an **existing** theme when an entry fits it. +- Surface your proposed grouping to the human and let them **rename, + merge, split, or reassign** before anything is final. Themes are the + human's call; you propose. + +### 3. Author a gist per touched theme + +For each theme in the plan, write the **gist** — one line, soft ceiling +~140 chars, saying what the theme *covers* (the shape of its knowledge), +not listing its entries. "hook mechanics: output channels, key names, +compliance wiring" — not "entry A; entry B". The gist tells a future +reader *whether to drill in*, nothing more. (Spec: `### Gist format`.) + +### 4. Present the plan — then stop + +Show the human, per theme: + +``` +Theme: <name> → .context/<noun>/<slug>.md (would create/append) + gist: <proposed one-line gist> + entries (N): + - [<ts>] <title> + - … +``` + +End with an explicit note: **"Dry run — nothing was moved or written. +The apply step (a later milestone) performs the move."** Do not edit any +file. Do not run an apply command (there is none yet). + +## Quality Checklist + +- [ ] Ran `ctx disclosure inspect --json`; did not hand-parse the file +- [ ] Every staged entry is assigned to exactly one theme in the plan +- [ ] Each theme has a one-line gist describing coverage (not a list) +- [ ] The plan was surfaced for the human to adjust themes +- [ ] **Nothing was edited, created, or moved** — output is a plan only diff --git a/internal/bootstrap/group.go b/internal/bootstrap/group.go index 9a3813ec4..9f8109513 100644 --- a/internal/bootstrap/group.go +++ b/internal/bootstrap/group.go @@ -14,6 +14,7 @@ import ( "github.com/ActiveMemory/ctx/internal/cli/connection" "github.com/ActiveMemory/ctx/internal/cli/convention" "github.com/ActiveMemory/ctx/internal/cli/decision" + "github.com/ActiveMemory/ctx/internal/cli/disclosure" "github.com/ActiveMemory/ctx/internal/cli/doctor" "github.com/ActiveMemory/ctx/internal/cli/dream" "github.com/ActiveMemory/ctx/internal/cli/drift" @@ -107,6 +108,7 @@ func artifacts() []registration { {task.Cmd, embedCmd.GroupArtifacts}, {convention.Cmd, embedCmd.GroupArtifacts}, {index.Cmd, embedCmd.GroupArtifacts}, + {disclosure.Cmd, embedCmd.GroupArtifacts}, {kb.Cmd, embedCmd.GroupArtifacts}, {handover.Cmd, embedCmd.GroupArtifacts}, } diff --git a/internal/cli/add/core/insert/insert.go b/internal/cli/add/core/insert/insert.go index ad07e0558..46597e413 100644 --- a/internal/cli/add/core/insert/insert.go +++ b/internal/cli/add/core/insert/insert.go @@ -17,12 +17,15 @@ import ( "github.com/ActiveMemory/ctx/internal/inspect" ) -// AfterHeader finds a header line and inserts content after it. +// AfterHeader finds a header line and inserts content after it, +// preserving any content that follows the insertion point. // // Skips blank lines and HTML comment blocks (<!-- ... -->) between the header // and the insertion point, so new entries land after index tables, format // guides, and other comment-wrapped metadata. Falls back to appending at the -// end if the header is not found. +// end if the header is not found. When content already follows the insertion +// point, the entry is separated from it by a "---" rule; when nothing +// follows, the entry is appended as-is. // // Parameters: // - content: Existing file content @@ -69,7 +72,23 @@ func AfterHeader(content, entry, header string) []byte { insertPoint = inspect.SkipWhitespace(content, insertPoint) } - return []byte(content[:insertPoint] + entry) + // Preserve whatever follows the insertion point. Returning + // content[:insertPoint]+entry truncated the file here, silently + // discarding the tail: on an entry-less knowledge file any section + // below the preamble was destroyed by an add. An empty tail keeps + // the original byte-for-byte output, so only the losing shape + // changes. See specs/fix-afterheader-tail-truncation.md. + tail := content[insertPoint:] + if tail == "" { + return []byte(content[:insertPoint] + entry) + } + + return []byte( + content[:insertPoint] + entry + + token.NewlineLF + token.Separator + + token.NewlineLF + token.NewlineLF + + tail, + ) } // AppendAtEnd appends an entry at the end of content. diff --git a/internal/cli/add/core/insert/insert_test.go b/internal/cli/add/core/insert/insert_test.go new file mode 100644 index 000000000..59d9bd496 --- /dev/null +++ b/internal/cli/add/core/insert/insert_test.go @@ -0,0 +1,109 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package insert_test + +import ( + "strings" + "testing" + + "github.com/ActiveMemory/ctx/internal/cli/add/core/insert" + cfgEntry "github.com/ActiveMemory/ctx/internal/config/entry" +) + +// Regression tests for specs/fix-afterheader-tail-truncation.md. +// +// AfterHeader returned content[:insertPoint] + entry — truncating the +// file at the insert point and discarding everything after it. Its sole +// caller is beforeFirstEntry's fallback, taken when a knowledge file has +// no "## [" entry, so `ctx learning add` against an entry-less file +// destroyed any content below the H1 header and its comment block. + +const preamble = `# Learnings + +<!-- +UPDATE WHEN: +- Discover a gotcha worth recording +--> + +` + +// tailSection is content that lives below the preamble of a file which +// has no "## [" entries — exactly the shape that triggers the fallback. +const tailSection = `## Notes + +- a hand-written section that must survive an add +` + +const addedEntry = "## [2026-07-17-120000] A brand new learning\n\n" + + "**Context**: freshly added.\n" + +const addedEntryHeader = "## [2026-07-17-120000]" + +// TestAfterHeader_PreservesTail is the bug: content below the preamble +// of an entry-less file must survive, with the new entry above it. +func TestAfterHeader_PreservesTail(t *testing.T) { + out := string(insert.AppendEntry( + []byte(preamble+tailSection), addedEntry, cfgEntry.Learning, "", + )) + + if !strings.Contains(out, "## Notes") { + t.Fatalf("tail section was destroyed by add; got:\n%s", out) + } + if !strings.Contains(out, "a hand-written section that must survive an add") { + t.Fatalf("tail body was destroyed by add; got:\n%s", out) + } + + entryIdx := strings.Index(out, addedEntryHeader) + tailIdx := strings.Index(out, "## Notes") + if entryIdx == -1 { + t.Fatalf("added entry missing; got:\n%s", out) + } + if entryIdx > tailIdx { + t.Errorf("entry landed below the tail (entry=%d tail=%d); "+ + "it must land directly after the preamble", entryIdx, tailIdx) + } +} + +// TestAfterHeader_EmptyTailUnchanged pins the shape that works today: +// an entry-less file with nothing after the comment block must produce +// byte-identical output to the pre-fix implementation (content + entry). +func TestAfterHeader_EmptyTailUnchanged(t *testing.T) { + out := string(insert.AppendEntry( + []byte(preamble), addedEntry, cfgEntry.Learning, "", + )) + + want := preamble + addedEntry + if out != want { + t.Errorf("empty-tail output changed.\n got: %q\nwant: %q", out, want) + } +} + +// TestBeforeFirstEntry_PrimaryPathUnchanged guards the common path: a +// file that already has entries still inserts before the first one and +// keeps every existing entry. +func TestBeforeFirstEntry_PrimaryPathUnchanged(t *testing.T) { + existing := preamble + + "## [2026-07-15-141726] an existing learning\n\n**Context**: old.\n" + + out := string(insert.AppendEntry( + []byte(existing), addedEntry, cfgEntry.Learning, "", + )) + + if !strings.Contains(out, "an existing learning") { + t.Fatalf("existing entry destroyed; got:\n%s", out) + } + + newIdx := strings.Index(out, addedEntryHeader) + oldIdx := strings.Index(out, "## [2026-07-15-141726]") + if newIdx == -1 || oldIdx == -1 { + t.Fatalf("an entry is missing; got:\n%s", out) + } + if newIdx > oldIdx { + t.Errorf("new entry must precede the existing one "+ + "(new=%d old=%d)", newIdx, oldIdx) + } +} diff --git a/internal/cli/add/core/insert/layout_convention_test.go b/internal/cli/add/core/insert/layout_convention_test.go new file mode 100644 index 000000000..cba909e8f --- /dev/null +++ b/internal/cli/add/core/insert/layout_convention_test.go @@ -0,0 +1,68 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package insert_test + +import ( + "strings" + "testing" + + "github.com/ActiveMemory/ctx/internal/cli/add/core/insert" + cfgEntry "github.com/ActiveMemory/ctx/internal/config/entry" +) + +// Layout proof for specs/progressive-disclosure.md (plan pd-m1, T12): +// CONVENTIONS. Unlike learnings/decisions, conventions append at EOF +// (AppendAtEnd) and their entries are `###` prose sections. So the design +// puts the staging zone in a trailing `## Recent` section (last in the +// file) — an EOF append lands the new section inside it, below `## Themes`. + +const rootConvention = `# Conventions + +<!-- convention format … --> + +## Themes + +- naming — file and symbol naming rules → [naming](conventions/naming.md) +- testing — colocation and integration split → [testing](conventions/testing.md) + +## Recent + +### an existing recent convention + +Some prose. +` + +const newConvention = "### a brand new convention\n\nMore prose.\n" + +func TestAdd_ConventionLandsInRecent(t *testing.T) { + out := string(insert.AppendEntry( + []byte(rootConvention), newConvention, cfgEntry.Convention, "", + )) + + // Themes section and its links must survive. + if !strings.Contains(out, "## Themes") || + !strings.Contains(out, "[naming](conventions/naming.md)") { + t.Fatalf("Themes section/link destroyed by add; result:\n%s", out) + } + + newIdx := strings.Index(out, "### a brand new convention") + recentIdx := strings.Index(out, "## Recent") + themesIdx := strings.Index(out, "## Themes") + if newIdx == -1 { + t.Fatalf("new convention missing; result:\n%s", out) + } + // The new section must land inside ## Recent — i.e. after both the + // Themes and the Recent headings. + if newIdx < recentIdx { + t.Errorf("convention landed ABOVE ## Recent (new=%d recent=%d); "+ + "it must land inside the trailing staging section", newIdx, recentIdx) + } + if newIdx < themesIdx { + t.Errorf("convention landed above ## Themes (new=%d themes=%d)", + newIdx, themesIdx) + } +} diff --git a/internal/cli/add/core/insert/layout_decision_test.go b/internal/cli/add/core/insert/layout_decision_test.go new file mode 100644 index 000000000..4180bbe9f --- /dev/null +++ b/internal/cli/add/core/insert/layout_decision_test.go @@ -0,0 +1,78 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package insert_test + +import ( + "strings" + "testing" + + "github.com/ActiveMemory/ctx/internal/cli/add/core/insert" + cfgEntry "github.com/ActiveMemory/ctx/internal/config/entry" +) + +// Layout proof for specs/progressive-disclosure.md (plan pd-m1, T11): +// DECISIONS. Decision() shares beforeFirstEntry with learnings, so the +// same premise applies — a new decision lands above ## Themes in both the +// populated- and empty-staging cases. + +const rootDecisionPopulated = `# Decisions + +<!-- DECISION FORMATS … --> + +## [2026-07-15-000000] an existing decision + +**Status**: Accepted + +--- + +## Themes + +- naming — singular CLI entity names → [naming](decisions/naming.md) +` + +const rootDecisionEmpty = `# Decisions + +<!-- DECISION FORMATS … --> + +## Themes + +- naming — singular CLI entity names → [naming](decisions/naming.md) +` + +const newDecision = "## [2026-07-17-120000] A brand new decision\n\n" + + "**Status**: Accepted\n" + +func assertDecisionAboveThemes(t *testing.T, out string) { + t.Helper() + if !strings.Contains(out, "## Themes") || + !strings.Contains(out, "[naming](decisions/naming.md)") { + t.Fatalf("Themes section/link destroyed by add; result:\n%s", out) + } + entryIdx := strings.Index(out, "## [2026-07-17-120000]") + themesIdx := strings.Index(out, "## Themes") + if entryIdx == -1 { + t.Fatalf("new decision missing; result:\n%s", out) + } + if entryIdx > themesIdx { + t.Errorf("decision landed below ## Themes (entry=%d themes=%d)", + entryIdx, themesIdx) + } +} + +func TestAdd_DecisionLandsAboveThemes_PopulatedStaging(t *testing.T) { + out := string(insert.AppendEntry( + []byte(rootDecisionPopulated), newDecision, cfgEntry.Decision, "", + )) + assertDecisionAboveThemes(t, out) +} + +func TestAdd_DecisionLandsAboveThemes_EmptyStaging(t *testing.T) { + out := string(insert.AppendEntry( + []byte(rootDecisionEmpty), newDecision, cfgEntry.Decision, "", + )) + assertDecisionAboveThemes(t, out) +} diff --git a/internal/cli/add/core/insert/layout_learning_test.go b/internal/cli/add/core/insert/layout_learning_test.go new file mode 100644 index 000000000..4a1c8021f --- /dev/null +++ b/internal/cli/add/core/insert/layout_learning_test.go @@ -0,0 +1,113 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package insert_test + +import ( + "strings" + "testing" + + "github.com/ActiveMemory/ctx/internal/cli/add/core/insert" + cfgEntry "github.com/ActiveMemory/ctx/internal/config/entry" +) + +// Layout proof for specs/progressive-disclosure.md (plan pd-m1, T10). +// +// The design's load-bearing premise is that `ctx learning add` needs ZERO +// code change under progressive disclosure, because its insert anchor +// always lands the new entry in the staging zone — above the `## Themes` +// section — including when staging is empty and the anchor falls back to +// AfterHeader. +// +// These tests are the milestone's measurement gate: if they fail, the +// premise is wrong and the spec's Layout section must be revisited via +// /ctx-plan rather than patched downstream. + +const themesSection = `## Themes + +- hooks — hook mechanics, output channels, compliance → [hooks](learnings/hooks.md) +- output — write/ taxonomy and emission style → [output](learnings/output.md) +` + +// rootPopulatedStaging: a Themes-bearing root that still has staged entries. +const rootPopulatedStaging = `# Learnings + +<!-- +UPDATE WHEN: +- Discover a gotcha worth recording +--> + +## [2026-07-15-141726] gosec G101 has two independent triggers + +**Context**: Removed the blanket exclusion. + +--- + +` + themesSection + +// rootEmptyStaging: a Themes-bearing root whose staging zone is empty — +// every entry has already been rolled out into its theme file. This is the +// steady state the design converges on, and the case that exercises the +// AfterHeader fallback. +const rootEmptyStaging = `# Learnings + +<!-- +UPDATE WHEN: +- Discover a gotcha worth recording +--> + +` + themesSection + +const newEntry = "## [2026-07-17-120000] A brand new learning\n\n" + + "**Context**: freshly added.\n" + +const newEntryHeader = "## [2026-07-17-120000]" + +// assertLandsAboveThemes asserts the two things the premise requires: the +// Themes section survives the add, and the new entry sits above it (in the +// staging zone). +func assertLandsAboveThemes(t *testing.T, out string) { + t.Helper() + + if !strings.Contains(out, "## Themes") { + t.Fatalf("Themes section was DESTROYED by add; result:\n%s", out) + } + if !strings.Contains(out, "[hooks](learnings/hooks.md)") { + t.Fatalf("theme gists/links were DESTROYED by add; result:\n%s", out) + } + + entryIdx := strings.Index(out, newEntryHeader) + themesIdx := strings.Index(out, "## Themes") + if entryIdx == -1 { + t.Fatalf("new entry missing from result:\n%s", out) + } + if entryIdx > themesIdx { + t.Errorf( + "new entry landed BELOW ## Themes (entry=%d themes=%d); "+ + "it must land in the staging zone above it", + entryIdx, themesIdx, + ) + } +} + +// TestAdd_LearningLandsAboveThemes_PopulatedStaging covers the common case: +// staging holds entries, so the anchor finds a `## [` and inserts before it. +func TestAdd_LearningLandsAboveThemes_PopulatedStaging(t *testing.T) { + out := string(insert.AppendEntry( + []byte(rootPopulatedStaging), newEntry, cfgEntry.Learning, "", + )) + assertLandsAboveThemes(t, out) +} + +// TestAdd_LearningLandsAboveThemes_EmptyStaging covers the steady state: +// staging is empty, so the anchor finds no `## [` and falls back to +// AfterHeader. The premise says the entry still lands above ## Themes. +func TestAdd_LearningLandsAboveThemes_EmptyStaging(t *testing.T) { + out := string(insert.AppendEntry( + []byte(rootEmptyStaging), newEntry, cfgEntry.Learning, "", + )) + assertLandsAboveThemes(t, out) +} diff --git a/internal/cli/add/core/insert/testmain_test.go b/internal/cli/add/core/insert/testmain_test.go new file mode 100644 index 000000000..192ce0250 --- /dev/null +++ b/internal/cli/add/core/insert/testmain_test.go @@ -0,0 +1,24 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package insert_test + +import ( + "os" + "testing" + + "github.com/ActiveMemory/ctx/internal/assets/read/lookup" +) + +// TestMain initializes the embedded asset lookup so desc.Text resolves +// real values. Without it every desc.Text call returns "", and +// strings.Index(s, "") == 0 silently rewrites the insert anchors into +// "match at offset 0" — tests would exercise a path production never +// takes, and pass for the wrong reason. +func TestMain(m *testing.M) { + lookup.Init() + os.Exit(m.Run()) +} diff --git a/internal/cli/disclosure/cmd/inspect/cmd.go b/internal/cli/disclosure/cmd/inspect/cmd.go new file mode 100644 index 000000000..b16795dec --- /dev/null +++ b/internal/cli/disclosure/cmd/inspect/cmd.go @@ -0,0 +1,43 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package inspect + +import ( + "github.com/spf13/cobra" + + "github.com/ActiveMemory/ctx/internal/assets/read/desc" + embedCmd "github.com/ActiveMemory/ctx/internal/config/embed/cmd" + "github.com/ActiveMemory/ctx/internal/config/embed/flag" + cFlag "github.com/ActiveMemory/ctx/internal/config/flag" + "github.com/ActiveMemory/ctx/internal/flagbind" +) + +// Cmd returns the "ctx disclosure inspect" command. +// +// It takes one positional argument (a canonical knowledge file) and +// reports its staged entries and current themes. Read-only; --json +// switches to machine-readable output. +// +// Returns: +// - *cobra.Command: configured inspect command with --json registered +func Cmd() *cobra.Command { + var jsonOutput bool + + short, long := desc.Command(embedCmd.DescKeyDisclosureInspect) + c := &cobra.Command{ + Use: embedCmd.UseDisclosureInspect, + Short: short, + Long: long, + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + return Run(cmd, args[0], jsonOutput) + }, + } + + flagbind.BoolFlag(c, &jsonOutput, cFlag.JSON, flag.DescKeyIndexJSON) + return c +} diff --git a/internal/cli/disclosure/cmd/inspect/doc.go b/internal/cli/disclosure/cmd/inspect/doc.go new file mode 100644 index 000000000..b426e2dd4 --- /dev/null +++ b/internal/cli/disclosure/cmd/inspect/doc.go @@ -0,0 +1,23 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +// Package inspect implements `ctx disclosure inspect <file>`: a +// read-only report of a canonical knowledge file's staged entries and +// current themes, for the progressive-disclosure dry-run pass. +// +// # Domain +// +// [Cmd] wires the cobra command; [Run] resolves the file's kind (via +// internal/disclosure.KindFor), reads it, builds an +// internal/disclosure.Inspection, and renders it through +// internal/write/disclosure. It writes nothing to the file. +// +// # Related packages +// +// - internal/disclosure — Parse/Inspect the root. +// - internal/write/disclosure — render the Inspection. +// - internal/err/disclosure — the not-a-knowledge-file error. +package inspect diff --git a/internal/cli/disclosure/cmd/inspect/run.go b/internal/cli/disclosure/cmd/inspect/run.go new file mode 100644 index 000000000..c725f1c90 --- /dev/null +++ b/internal/cli/disclosure/cmd/inspect/run.go @@ -0,0 +1,51 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package inspect + +import ( + "path/filepath" + + "github.com/spf13/cobra" + + "github.com/ActiveMemory/ctx/internal/disclosure" + errDisc "github.com/ActiveMemory/ctx/internal/err/disclosure" + errFs "github.com/ActiveMemory/ctx/internal/err/fs" + "github.com/ActiveMemory/ctx/internal/io" + writeDisc "github.com/ActiveMemory/ctx/internal/write/disclosure" +) + +// Run executes the inspect command: resolve the file's kind, read it, +// and report its staged entries and current themes. It never writes. +// +// Parameters: +// - cmd: Cobra command for the output stream +// - path: path to the canonical knowledge file +// - jsonOutput: if true, emit JSON instead of a human summary +// +// Returns: +// - error: NotAKnowledgeFile when path is not a canonical file, a +// path-bearing read error, or a JSON-marshal error +func Run(cmd *cobra.Command, path string, jsonOutput bool) error { + kind, ok := disclosure.KindFor(filepath.Base(path)) + if !ok { + cmd.SilenceUsage = true + return errDisc.NotAKnowledgeFile(path) + } + + content, readErr := io.SafeReadUserFile(filepath.Clean(path)) + if readErr != nil { + cmd.SilenceUsage = true + return errFs.FileRead(path, readErr) + } + + insp := disclosure.Inspect(string(content), kind) + if jsonOutput { + return writeDisc.JSON(cmd, insp) + } + writeDisc.Human(cmd, insp) + return nil +} diff --git a/internal/cli/disclosure/cmd/inspect/run_test.go b/internal/cli/disclosure/cmd/inspect/run_test.go new file mode 100644 index 000000000..debf3a3e4 --- /dev/null +++ b/internal/cli/disclosure/cmd/inspect/run_test.go @@ -0,0 +1,118 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package inspect_test + +import ( + "bytes" + "encoding/json" + "errors" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/spf13/cobra" + + "github.com/ActiveMemory/ctx/internal/assets/read/lookup" + "github.com/ActiveMemory/ctx/internal/cli/disclosure/cmd/inspect" + "github.com/ActiveMemory/ctx/internal/disclosure" + errDisc "github.com/ActiveMemory/ctx/internal/err/disclosure" +) + +// TestMain initializes the asset lookup so desc.Text resolves the output +// labels and error text (see the "uninitialized desc.Text" learning). +func TestMain(m *testing.M) { + lookup.Init() + os.Exit(m.Run()) +} + +const fixtureRoot = "# Learnings\n\n<!-- guide -->\n\n" + + "## [2026-07-15-120000] a staged entry\n\n**Context**: x.\n\n---\n\n" + + "## Themes\n\n- hooks — hook mechanics → [hooks](learnings/hooks.md)\n" + +// writeFixture creates <dir>/<name> with content and returns its path. +func writeFixture(t *testing.T, name, content string) string { + t.Helper() + path := filepath.Join(t.TempDir(), name) + if err := os.WriteFile(path, []byte(content), 0o600); err != nil { + t.Fatalf("write fixture: %v", err) + } + return path +} + +// run invokes inspect.Run with an output buffer and returns stdout + err. +func run(t *testing.T, path string, jsonOut bool) (string, error) { + t.Helper() + var out bytes.Buffer + cmd := &cobra.Command{} + cmd.SetOut(&out) + cmd.SetErr(&out) + err := inspect.Run(cmd, path, jsonOut) + return out.String(), err +} + +// T05: human output lists the kind, staged entries, and themes. +func TestRun_HumanOutput(t *testing.T) { + path := writeFixture(t, "LEARNINGS.md", fixtureRoot) + out, err := run(t, path, false) + if err != nil { + t.Fatalf("Run error = %v", err) + } + for _, want := range []string{ + "learning", "Staged (1)", "a staged entry", "Themes (1)", + "hooks", "learnings/hooks.md", + } { + if !strings.Contains(out, want) { + t.Errorf("human output missing %q; got:\n%s", want, out) + } + } +} + +// T06: --json output decodes into an Inspection with expected values. +func TestRun_JSONOutput(t *testing.T) { + path := writeFixture(t, "LEARNINGS.md", fixtureRoot) + out, err := run(t, path, true) + if err != nil { + t.Fatalf("Run error = %v", err) + } + var got disclosure.Inspection + if uErr := json.Unmarshal([]byte(out), &got); uErr != nil { + t.Fatalf("output is not valid JSON: %v\n%s", uErr, out) + } + if got.Kind != "learning" { + t.Errorf("Kind = %q, want learning", got.Kind) + } + if len(got.Staging) != 1 || got.Staging[0].Title != "a staged entry" { + t.Errorf("Staging = %+v", got.Staging) + } + if len(got.Themes) != 1 || got.Themes[0].Link != "learnings/hooks.md" { + t.Errorf("Themes = %+v", got.Themes) + } +} + +// T07: inspect writes nothing — the file is byte-identical after. +func TestRun_WritesNothing(t *testing.T) { + path := writeFixture(t, "DECISIONS.md", + strings.Replace(fixtureRoot, "# Learnings", "# Decisions", 1)) + before, _ := os.ReadFile(path) + if _, err := run(t, path, false); err != nil { + t.Fatalf("Run error = %v", err) + } + after, _ := os.ReadFile(path) + if !bytes.Equal(before, after) { + t.Errorf("inspect modified the file:\nbefore=%q\nafter=%q", before, after) + } +} + +// T08: a non-knowledge file is rejected with the typed sentinel. +func TestRun_RejectsNonKnowledgeFile(t *testing.T) { + path := writeFixture(t, "README.md", "# readme\n") + _, err := run(t, path, false) + if !errors.Is(err, errDisc.ErrNotAKnowledgeFile) { + t.Errorf("Run(README.md) err = %v, want ErrNotAKnowledgeFile", err) + } +} diff --git a/internal/cli/disclosure/disclosure.go b/internal/cli/disclosure/disclosure.go new file mode 100644 index 000000000..a7b4d7589 --- /dev/null +++ b/internal/cli/disclosure/disclosure.go @@ -0,0 +1,26 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + "github.com/spf13/cobra" + + "github.com/ActiveMemory/ctx/internal/cli/disclosure/cmd/inspect" + "github.com/ActiveMemory/ctx/internal/cli/parent" + "github.com/ActiveMemory/ctx/internal/config/embed/cmd" +) + +// Cmd returns the `ctx disclosure` parent command, which inspects the +// canonical knowledge files under progressive disclosure. +// +// Returns: +// - *cobra.Command: disclosure parent with the inspect subcommand +func Cmd() *cobra.Command { + return parent.Cmd(cmd.DescKeyDisclosure, cmd.UseDisclosure, + inspect.Cmd(), + ) +} diff --git a/internal/cli/disclosure/doc.go b/internal/cli/disclosure/doc.go new file mode 100644 index 000000000..3e803f700 --- /dev/null +++ b/internal/cli/disclosure/doc.go @@ -0,0 +1,22 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +// Package disclosure assembles the `ctx disclosure` command group for +// progressive disclosure of the canonical knowledge files (see +// specs/progressive-disclosure.md). +// +// [Cmd] returns the parent command with its subcommand tree. +// +// Subcommands: +// +// - ctx disclosure inspect <file>: read-only report of a root's +// staged entries and current themes, which the dry-run digesting +// pass consumes. The mover (apply) subcommands arrive in a later +// milestone. +// +// The parse/inspect domain logic lives in internal/disclosure; output +// rendering in internal/write/disclosure. +package disclosure diff --git a/internal/compliance/disclosure_test.go b/internal/compliance/disclosure_test.go new file mode 100644 index 000000000..923eba347 --- /dev/null +++ b/internal/compliance/disclosure_test.go @@ -0,0 +1,81 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package compliance + +import ( + "errors" + "os" + "path/filepath" + "testing" + + cfgCtx "github.com/ActiveMemory/ctx/internal/config/ctx" + cfgDisc "github.com/ActiveMemory/ctx/internal/config/disclosure" + "github.com/ActiveMemory/ctx/internal/disclosure" + errDisc "github.com/ActiveMemory/ctx/internal/err/disclosure" +) + +// TestDisclosureInvariants runs the progressive-disclosure guards and +// invariants against this repo's own .context/ knowledge files. The +// tree is not yet migrated, so every check passes vacuously — this is +// the guard that no canonical file drifts into a shape the guards would +// reject before migration even begins. +// +// It is proven-both-ways below (TestDisclosureInvariants_PlantedViolation): +// a check that only ever passes proves nothing. +func TestDisclosureInvariants(t *testing.T) { + root, err := filepath.Abs(filepath.Join("..", "..")) + if err != nil { + t.Fatalf("resolve repo root: %v", err) + } + ctxDir := filepath.Join(root, ".context") + + cases := []struct { + file string + themeDir string + kind disclosure.Kind + }{ + {cfgCtx.Learning, cfgDisc.ThemeDirLearning, disclosure.KindLearning}, + {cfgCtx.Decision, cfgDisc.ThemeDirDecision, disclosure.KindDecision}, + {cfgCtx.Convention, cfgDisc.ThemeDirConvention, disclosure.KindConvention}, + } + + for _, tc := range cases { + t.Run(tc.file, func(t *testing.T) { + data, readErr := os.ReadFile(filepath.Join(ctxDir, tc.file)) + if readErr != nil { + t.Fatalf("read %s: %v", tc.file, readErr) + } + r := disclosure.Parse(string(data), tc.kind) + themeDir := filepath.Join(ctxDir, tc.themeDir) + + if vErr := disclosure.Validate(r); vErr != nil { + t.Errorf("Validate(%s) = %v, want nil", tc.file, vErr) + } + if pErr := disclosure.CheckPairing(r, themeDir); pErr != nil { + t.Errorf("CheckPairing(%s) = %v, want nil", tc.file, pErr) + } + if uErr := disclosure.CheckUniqueness(r, themeDir); uErr != nil { + t.Errorf("CheckUniqueness(%s) = %v, want nil", tc.file, uErr) + } + if lErr := disclosure.CheckLinks(r, ctxDir); lErr != nil { + t.Errorf("CheckLinks(%s) = %v, want nil", tc.file, lErr) + } + }) + } +} + +// TestDisclosureInvariants_PlantedViolation proves the guard actually +// fires: a root with two ## Themes sections must be refused. +func TestDisclosureInvariants_PlantedViolation(t *testing.T) { + planted := "# Learnings\n\n## Themes\n\n- a — g → [a](learnings/a.md)\n\n" + + "## Themes\n\n- b — g → [b](learnings/b.md)\n" + + err := disclosure.Validate(disclosure.Parse(planted, disclosure.KindLearning)) + if !errors.Is(err, errDisc.ErrMultipleThemes) { + t.Errorf("Validate(planted double-Themes) = %v, want ErrMultipleThemes", err) + } +} diff --git a/internal/config/disclosure/disclosure.go b/internal/config/disclosure/disclosure.go new file mode 100644 index 000000000..b4aa59e08 --- /dev/null +++ b/internal/config/disclosure/disclosure.go @@ -0,0 +1,46 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +// Structural vocabulary for progressive-disclosure roots: the headings +// that delimit a root's regions, the line prefixes that mark entries, +// and the per-kind theme-file directories. +const ( + // HeadingThemes delimits the themes region of a root — the per-theme + // gists and links. In entry-based roots (LEARNINGS/DECISIONS) it also + // marks the lower bound of the staging zone: no entry may appear below + // it. + HeadingThemes = "## Themes" + + // HeadingRecent delimits the staging zone of a CONVENTIONS root. + // Conventions append at EOF and their entries are prose sections, so + // their staging needs an explicit trailing heading that "## Themes" + // cannot provide. + HeadingRecent = "## Recent" + + // EntryLinePrefix marks a timestamped entry heading ("## [ts] Title") + // in LEARNINGS/DECISIONS. + EntryLinePrefix = "## [" + + // ConventionLinePrefix marks a prose section heading in CONVENTIONS. + ConventionLinePrefix = "### " + + // IDSeparator joins the timestamp and title of an entry identity. A + // NUL never appears in a heading line, so entry text cannot forge it. + IDSeparator = "\x00" + + // LinkOpen is the "](" that separates a markdown link's label from its + // target; a theme gist's link is the "(...)" following it. + LinkOpen = "](" + + // ThemeDirLearning, ThemeDirDecision, and ThemeDirConvention name the + // per-kind subdirectories of the context directory that hold theme + // files (<theme>.md), reachable only via the root's links. + ThemeDirLearning = "learnings" + ThemeDirDecision = "decisions" + ThemeDirConvention = "conventions" +) diff --git a/internal/config/disclosure/disclosure_test.go b/internal/config/disclosure/disclosure_test.go new file mode 100644 index 000000000..09a0c02e6 --- /dev/null +++ b/internal/config/disclosure/disclosure_test.go @@ -0,0 +1,26 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure_test + +import ( + "testing" + + "github.com/ActiveMemory/ctx/internal/config/disclosure" +) + +// T01: the structural vocabulary is fixed and load-bearing — the layout +// proofs and Validate key on these exact strings. +func TestHeadingConstants(t *testing.T) { + if disclosure.HeadingThemes != "## Themes" { + t.Errorf("HeadingThemes = %q, want %q", + disclosure.HeadingThemes, "## Themes") + } + if disclosure.HeadingRecent != "## Recent" { + t.Errorf("HeadingRecent = %q, want %q", + disclosure.HeadingRecent, "## Recent") + } +} diff --git a/internal/config/disclosure/doc.go b/internal/config/disclosure/doc.go new file mode 100644 index 000000000..08924a3c9 --- /dev/null +++ b/internal/config/disclosure/doc.go @@ -0,0 +1,28 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +// Package disclosure holds the structural vocabulary for +// progressive-disclosure roots (see specs/progressive-disclosure.md). +// +// # Domain +// +// A bounded root is delimited by ATX headings whose exact text is +// load-bearing: [HeadingThemes] ("## Themes") marks the themes region +// and, for entry files, the lower bound of the staging zone; +// [HeadingRecent] ("## Recent") marks the staging zone of a CONVENTIONS +// root, which appends at EOF and cannot reuse "## Themes" (a "###" prose +// section would nest ambiguously inside it). [ThemeDirLearning], +// [ThemeDirDecision], and [ThemeDirConvention] name the per-kind +// subdirectories of the context directory that hold theme files. +// +// These constants are a single source of truth: the parser +// (internal/disclosure), the validate precondition, and the add-path +// layout proofs all key on these exact strings. +// +// # Concurrency +// +// Compile-time constants; safe for concurrent use. +package disclosure diff --git a/internal/config/embed/cmd/base.go b/internal/config/embed/cmd/base.go index 09eb2058a..c5ac1d289 100644 --- a/internal/config/embed/cmd/base.go +++ b/internal/config/embed/cmd/base.go @@ -14,6 +14,11 @@ const ( UseChange = "change" // UseCompact is the cobra Use string for the compact command. UseCompact = "compact" + // UseDisclosure is the cobra Use string for the disclosure command group. + UseDisclosure = "disclosure" + // UseDisclosureInspect is the cobra Use string for the disclosure + // inspect subcommand. + UseDisclosureInspect = "inspect [FILE]" // UseDoctor is the cobra Use string for the doctor command. UseDoctor = "doctor" // UseDrift is the cobra Use string for the drift command. @@ -90,6 +95,11 @@ const ( DescKeyInitialize = "initialize" // DescKeyIndex is the description key for the index command. DescKeyIndex = "index" + // DescKeyDisclosure is the description key for the disclosure group. + DescKeyDisclosure = "disclosure" + // DescKeyDisclosureInspect is the description key for the disclosure + // inspect subcommand. + DescKeyDisclosureInspect = "disclosure-inspect" // DescKeyLoad is the description key for the load command. DescKeyLoad = "load" // DescKeyLoop is the description key for the loop command. diff --git a/internal/config/embed/text/disclosure.go b/internal/config/embed/text/disclosure.go new file mode 100644 index 000000000..118d55543 --- /dev/null +++ b/internal/config/embed/text/disclosure.go @@ -0,0 +1,58 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package text + +// DescKeys for progressive-disclosure output labels. Each value is the +// key of an entry in commands/text/write.yaml. +const ( + // DescKeyWriteDisclosureKind labels the inspected root's kind. + DescKeyWriteDisclosureKind = "write.disclosure-kind" + // DescKeyWriteDisclosureStagedHeader heads the staged-entry list. + DescKeyWriteDisclosureStagedHeader = "write.disclosure-staged-header" + // DescKeyWriteDisclosureStagedLine formats one staged entry. + DescKeyWriteDisclosureStagedLine = "write.disclosure-staged-line" + // DescKeyWriteDisclosureThemesHeader heads the themes list. + DescKeyWriteDisclosureThemesHeader = "write.disclosure-themes-header" + // DescKeyWriteDisclosureThemeLine formats one theme. + DescKeyWriteDisclosureThemeLine = "write.disclosure-theme-line" + // DescKeyWriteDisclosureNone marks an empty list. + DescKeyWriteDisclosureNone = "write.disclosure-none" +) + +// DescKeys for progressive-disclosure guard and invariant errors. +// Each value is the key of an entry in commands/text/errors.yaml; the +// DescKey <-> YAML mapping is a bijection enforced by +// internal/audit.TestDescKeyYAMLLinkage. +const ( + // DescKeyErrDisclosureBrokenThemeLink: a theme link in the root + // resolves to a nonexistent path. + DescKeyErrDisclosureBrokenThemeLink = "err.disclosure.broken-theme-link" + // DescKeyErrDisclosureDuplicateEntry: an entry appears in more than + // one place (staging and/or multiple theme files). + DescKeyErrDisclosureDuplicateEntry = "err.disclosure.duplicate-entry" + // DescKeyErrDisclosureEntryBelowThemes: a "## [" entry sits below the + // "## Themes" section instead of in the staging zone above it. + DescKeyErrDisclosureEntryBelowThemes = "err.disclosure.entry-below-themes" + // DescKeyErrDisclosureMissingThemeFile: a theme gist in the root has + // no matching theme file. + DescKeyErrDisclosureMissingThemeFile = "err.disclosure.missing-theme-file" + // DescKeyErrDisclosureMultipleThemes: the root has more than one + // "## Themes" section. + DescKeyErrDisclosureMultipleThemes = "err.disclosure.multiple-themes" + // DescKeyErrDisclosureNotKnowledgeFileMsg: the sentinel text for a + // file that is not a canonical knowledge file. + DescKeyErrDisclosureNotKnowledgeFileMsg = "err.disclosure.not-knowledge-file-msg" + // DescKeyErrDisclosureNotKnowledgeFile: the format wrapper naming the + // offending path for a not-a-knowledge-file error. + DescKeyErrDisclosureNotKnowledgeFile = "err.disclosure.not-knowledge-file" + // DescKeyErrDisclosureOrphanThemeFile: a theme file has no matching + // gist in the root. + DescKeyErrDisclosureOrphanThemeFile = "err.disclosure.orphan-theme-file" + // DescKeyErrDisclosureStagingUnparsable: the staging zone could not + // be parsed into discrete entries. + DescKeyErrDisclosureStagingUnparsable = "err.disclosure.staging-unparsable" +) diff --git a/internal/disclosure/collect.go b/internal/disclosure/collect.go new file mode 100644 index 000000000..22862acd4 --- /dev/null +++ b/internal/disclosure/collect.go @@ -0,0 +1,84 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + "os" + "path/filepath" + "strings" + + cfgDisc "github.com/ActiveMemory/ctx/internal/config/disclosure" + cfgFile "github.com/ActiveMemory/ctx/internal/config/file" + "github.com/ActiveMemory/ctx/internal/heading" +) + +// gistBasenames returns the set of theme-file basenames the root's gists +// link to. +// +// Parameters: +// - root: the parsed root +// +// Returns: +// - map[string]bool: set of basenames (e.g. "hooks.md") +func gistBasenames(root Root) map[string]bool { + names := map[string]bool{} + for _, t := range root.Themes { + if t.Link != "" { + names[filepath.Base(t.Link)] = true + } + } + return names +} + +// themeFiles returns the full paths of the .md files in themeDir. A +// missing directory is not an error — it means the root is not yet +// migrated, so there are no theme files. +// +// Parameters: +// - themeDir: directory to scan +// +// Returns: +// - []string: full paths of theme files (nil when the dir is absent) +// - error: a read error other than "does not exist" +func themeFiles(themeDir string) ([]string, error) { + entries, err := os.ReadDir(themeDir) + if err != nil { + if os.IsNotExist(err) { + return nil, nil + } + return nil, err + } + var paths []string + for _, e := range entries { + if !e.IsDir() && strings.HasSuffix(e.Name(), cfgFile.ExtMarkdown) { + paths = append(paths, filepath.Join(themeDir, e.Name())) + } + } + return paths, nil +} + +// entryIDs returns the identity of every "## [ts] Title" entry in +// content, in order. Identity is timestamp+title, not timestamp alone: +// two entries added in the same second share a timestamp but are +// distinct entries (observed in LEARNINGS.md). +// +// Parameters: +// - content: markdown to scan for entry blocks +// +// Returns: +// - []string: entry identities, nil if none +func entryIDs(content string) []string { + blocks := heading.ParseEntryBlocks(content) + if len(blocks) == 0 { + return nil + } + ids := make([]string, 0, len(blocks)) + for _, b := range blocks { + ids = append(ids, b.Entry.Timestamp+cfgDisc.IDSeparator+b.Entry.Title) + } + return ids +} diff --git a/internal/disclosure/doc.go b/internal/disclosure/doc.go new file mode 100644 index 000000000..09e2b0bd0 --- /dev/null +++ b/internal/disclosure/doc.go @@ -0,0 +1,46 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +// Package disclosure implements progressive disclosure for ctx's +// canonical knowledge files — LEARNINGS.md, DECISIONS.md, and +// CONVENTIONS.md (see specs/progressive-disclosure.md). +// +// A canonical file grows without bound while its entries stay valid, so +// an agent that reads the whole file to understand the system eventually +// exhausts its context window. Progressive disclosure turns each file +// into a BOUNDED ROOT: a compact `## Themes` section (a "just enough" +// gist per theme plus a link to a theme file) alongside a staging zone +// of recent, not-yet-digested entries. Entry bodies roll out into +// per-theme files reachable only via the root's links; the agent reads +// the bounded root and drills into a theme file on demand. +// +// # What this package provides (milestone 1) +// +// [Parse] splits a root into its regions without normalizing a byte, so +// [Root.Reconstruct] returns the input exactly — the digesting pass must +// see verbatim bodies. [Validate] is the fail-loud precondition +// (zero-or-one `## Themes`, no entry below it, staging parses into +// discrete entries) that refuses a malformed root rather than regenerate +// from "what it recognized" — the failure mode behind the historical +// clobber bug. [CheckPairing], [CheckUniqueness], and [CheckLinks] are +// the cross-file invariants: they keep the root ↔ theme-file link graph +// 1:1 and every entry in exactly one place. +// +// The digesting pass that moves bodies and writes gists is NOT here; it +// is an agent-driven, human-gated skill in a later milestone, built on +// these guards. Nothing in this package writes a knowledge file. +// +// # Related packages +// +// - internal/heading — parses the "## [ts] Title" entry blocks the +// staging and uniqueness checks rely on. +// - internal/config/disclosure — the structural heading vocabulary. +// - internal/err/disclosure — the guard and invariant sentinels. +// +// # Concurrency +// +// Pure data plus read-only theme-file stat/read; callers own all writes. +package disclosure diff --git a/internal/disclosure/inspect.go b/internal/disclosure/inspect.go new file mode 100644 index 000000000..ee83be88a --- /dev/null +++ b/internal/disclosure/inspect.go @@ -0,0 +1,54 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + "github.com/ActiveMemory/ctx/internal/heading" +) + +// StagedEntries returns the un-digested entries in a root's staging zone, +// in file order. Conventions have no "## [" entries, so this is empty for +// them (their digestion is a later milestone). +// +// Parameters: +// - root: the parsed root +// +// Returns: +// - []StagedEntry: the staged entries, nil when staging holds none +func StagedEntries(root Root) []StagedEntry { + blocks := heading.ParseEntryBlocks(root.Staging) + if len(blocks) == 0 { + return nil + } + entries := make([]StagedEntry, 0, len(blocks)) + for _, b := range blocks { + entries = append(entries, StagedEntry{ + Timestamp: b.Entry.Timestamp, + Title: b.Entry.Title, + }) + } + return entries +} + +// Inspect parses content as a root of kind k and returns the read-only +// view the dry-run pass consumes: kind name, staged entries, and current +// themes. Like Parse, it is total. +// +// Parameters: +// - content: the full root file content +// - k: which canonical file this is +// +// Returns: +// - Inspection: the structured read-only view +func Inspect(content string, k Kind) Inspection { + root := Parse(content, k) + return Inspection{ + Kind: k.String(), + Staging: StagedEntries(root), + Themes: root.Themes, + } +} diff --git a/internal/disclosure/inspect_test.go b/internal/disclosure/inspect_test.go new file mode 100644 index 000000000..6c4231b02 --- /dev/null +++ b/internal/disclosure/inspect_test.go @@ -0,0 +1,75 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure_test + +import ( + "testing" + + "github.com/ActiveMemory/ctx/internal/disclosure" +) + +// T01: KindFor maps the three canonical filenames and rejects others. +func TestKindFor(t *testing.T) { + cases := []struct { + name string + wantOK bool + wantStr string + }{ + {"LEARNINGS.md", true, "learning"}, + {"DECISIONS.md", true, "decision"}, + {"CONVENTIONS.md", true, "convention"}, + {"README.md", false, ""}, + {"learnings.md", false, ""}, // exact match only + {"", false, ""}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + k, ok := disclosure.KindFor(tc.name) + if ok != tc.wantOK { + t.Fatalf("KindFor(%q) ok = %v, want %v", tc.name, ok, tc.wantOK) + } + if ok && k.String() != tc.wantStr { + t.Errorf("KindFor(%q).String() = %q, want %q", + tc.name, k.String(), tc.wantStr) + } + }) + } +} + +// T02: StagedEntries lists the staging zone's entries in order, and is +// empty when staging holds none. +func TestStagedEntries(t *testing.T) { + populated := disclosure.Parse(entryMigratedPopulated, disclosure.KindLearning) + got := disclosure.StagedEntries(populated) + if len(got) != 1 { + t.Fatalf("StagedEntries = %d entries, want 1", len(got)) + } + if got[0].Timestamp != "2026-07-15-120000" || + got[0].Title != "a staged entry" { + t.Errorf("StagedEntries[0] = %+v, want ts/title of the staged entry", got[0]) + } + + empty := disclosure.Parse(entryMigratedEmpty, disclosure.KindLearning) + if s := disclosure.StagedEntries(empty); s != nil { + t.Errorf("StagedEntries(empty staging) = %+v, want nil", s) + } +} + +// T03: Inspect assembles kind + staging + themes consistent with Parse. +func TestInspect(t *testing.T) { + insp := disclosure.Inspect(entryMigratedPopulated, disclosure.KindLearning) + + if insp.Kind != "learning" { + t.Errorf("Inspect.Kind = %q, want learning", insp.Kind) + } + if len(insp.Staging) != 1 || insp.Staging[0].Title != "a staged entry" { + t.Errorf("Inspect.Staging = %+v, want the one staged entry", insp.Staging) + } + if len(insp.Themes) != 1 || insp.Themes[0].Link != "learnings/hooks.md" { + t.Errorf("Inspect.Themes = %+v, want the one theme", insp.Themes) + } +} diff --git a/internal/disclosure/invariant.go b/internal/disclosure/invariant.go new file mode 100644 index 000000000..0495a5b85 --- /dev/null +++ b/internal/disclosure/invariant.go @@ -0,0 +1,110 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + "os" + "path/filepath" + + errDisc "github.com/ActiveMemory/ctx/internal/err/disclosure" + internalIo "github.com/ActiveMemory/ctx/internal/io" +) + +// CheckPairing verifies the root's gists and the theme files are 1:1: no +// theme file the root cannot reach, and no gist pointing at a file that +// does not exist. +// +// An un-migrated root (no gists) with no theme directory is vacuously +// paired. +// +// Parameters: +// - root: the parsed root (its Themes supply the gist links) +// - themeDir: directory holding this kind's theme files +// +// Returns: +// - error: ErrOrphanThemeFile, ErrMissingThemeFile, a read error, or nil +func CheckPairing(root Root, themeDir string) error { + gists := gistBasenames(root) + files, err := themeFiles(themeDir) + if err != nil { + return err + } + + fileSet := map[string]bool{} + for _, f := range files { + name := filepath.Base(f) + fileSet[name] = true + if !gists[name] { + return errDisc.ErrOrphanThemeFile + } + } + for g := range gists { + if !fileSet[g] { + return errDisc.ErrMissingThemeFile + } + } + return nil +} + +// CheckUniqueness verifies every entry lives in exactly one place: the +// staging zone or a single theme file, never two. Entry identity is its +// timestamp+title. +// +// Parameters: +// - root: the parsed root (its Staging supplies staged entries) +// - themeDir: directory holding this kind's theme files +// +// Returns: +// - error: ErrDuplicateEntry, a read error, or nil +func CheckUniqueness(root Root, themeDir string) error { + seen := map[string]bool{} + for _, id := range entryIDs(root.Staging) { + if seen[id] { + return errDisc.ErrDuplicateEntry + } + seen[id] = true + } + + files, err := themeFiles(themeDir) + if err != nil { + return err + } + for _, f := range files { + data, readErr := internalIo.SafeReadUserFile(f) + if readErr != nil { + return readErr + } + for _, id := range entryIDs(string(data)) { + if seen[id] { + return errDisc.ErrDuplicateEntry + } + seen[id] = true + } + } + return nil +} + +// CheckLinks verifies every theme link in the root resolves to a path +// that exists, relative to the context directory. +// +// Parameters: +// - root: the parsed root (its Themes supply the links) +// - ctxDir: the context directory the links are relative to +// +// Returns: +// - error: ErrBrokenThemeLink, or nil +func CheckLinks(root Root, ctxDir string) error { + for _, t := range root.Themes { + if t.Link == "" { + continue + } + if _, statErr := os.Stat(filepath.Join(ctxDir, t.Link)); statErr != nil { + return errDisc.ErrBrokenThemeLink + } + } + return nil +} diff --git a/internal/disclosure/invariant_test.go b/internal/disclosure/invariant_test.go new file mode 100644 index 000000000..53a00a31a --- /dev/null +++ b/internal/disclosure/invariant_test.go @@ -0,0 +1,127 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure_test + +import ( + "errors" + "os" + "path/filepath" + "testing" + + "github.com/ActiveMemory/ctx/internal/disclosure" + errDisc "github.com/ActiveMemory/ctx/internal/err/disclosure" +) + +// writeThemeFile creates dir/name with content and returns dir. +func writeThemeFile(t *testing.T, dir, name, content string) { + t.Helper() + if mkErr := os.MkdirAll(dir, 0o750); mkErr != nil { + t.Fatalf("mkdir %s: %v", dir, mkErr) + } + if wErr := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o600); wErr != nil { + t.Fatalf("write %s: %v", name, wErr) + } +} + +func themeWith(name, link string) disclosure.Root { + return disclosure.Root{ + Kind: disclosure.KindLearning, + HasThemes: true, + Themes: []disclosure.Theme{{Name: name, Link: link}}, + } +} + +// T07: gists <-> theme files must be 1:1. +func TestInvariant_Pairing(t *testing.T) { + t.Run("1:1 passes", func(t *testing.T) { + dir := filepath.Join(t.TempDir(), "learnings") + writeThemeFile(t, dir, "hooks.md", "# hooks\n") + root := themeWith("hooks", "learnings/hooks.md") + if err := disclosure.CheckPairing(root, dir); err != nil { + t.Errorf("CheckPairing = %v, want nil", err) + } + }) + + t.Run("orphan theme file", func(t *testing.T) { + dir := filepath.Join(t.TempDir(), "learnings") + writeThemeFile(t, dir, "hooks.md", "# hooks\n") + writeThemeFile(t, dir, "extra.md", "# extra\n") // no gist points here + root := themeWith("hooks", "learnings/hooks.md") + if err := disclosure.CheckPairing(root, dir); !errors.Is(err, errDisc.ErrOrphanThemeFile) { + t.Errorf("CheckPairing = %v, want ErrOrphanThemeFile", err) + } + }) + + t.Run("gist with no file", func(t *testing.T) { + dir := filepath.Join(t.TempDir(), "learnings") // dir absent + root := themeWith("hooks", "learnings/hooks.md") + if err := disclosure.CheckPairing(root, dir); !errors.Is(err, errDisc.ErrMissingThemeFile) { + t.Errorf("CheckPairing = %v, want ErrMissingThemeFile", err) + } + }) + + t.Run("vacuous when un-migrated", func(t *testing.T) { + dir := filepath.Join(t.TempDir(), "learnings") // absent + root := disclosure.Root{Kind: disclosure.KindLearning} + if err := disclosure.CheckPairing(root, dir); err != nil { + t.Errorf("CheckPairing = %v, want nil (0<->0)", err) + } + }) +} + +// T08: every entry lives in exactly one place. +func TestInvariant_Uniqueness(t *testing.T) { + const entry = "## [2026-07-15-120000] the entry\n\n**Context**: x.\n" + + t.Run("single occurrence passes", func(t *testing.T) { + dir := filepath.Join(t.TempDir(), "learnings") + writeThemeFile(t, dir, "hooks.md", entry) + root := disclosure.Root{Kind: disclosure.KindLearning} // empty staging + if err := disclosure.CheckUniqueness(root, dir); err != nil { + t.Errorf("CheckUniqueness = %v, want nil", err) + } + }) + + t.Run("dup across staging and theme file", func(t *testing.T) { + dir := filepath.Join(t.TempDir(), "learnings") + writeThemeFile(t, dir, "hooks.md", entry) + root := disclosure.Root{Kind: disclosure.KindLearning, Staging: entry} + if err := disclosure.CheckUniqueness(root, dir); !errors.Is(err, errDisc.ErrDuplicateEntry) { + t.Errorf("CheckUniqueness = %v, want ErrDuplicateEntry", err) + } + }) + + t.Run("dup across two theme files", func(t *testing.T) { + dir := filepath.Join(t.TempDir(), "learnings") + writeThemeFile(t, dir, "hooks.md", entry) + writeThemeFile(t, dir, "output.md", entry) // same entry in two files + root := disclosure.Root{Kind: disclosure.KindLearning} + if err := disclosure.CheckUniqueness(root, dir); !errors.Is(err, errDisc.ErrDuplicateEntry) { + t.Errorf("CheckUniqueness = %v, want ErrDuplicateEntry", err) + } + }) +} + +// T09: every theme link must resolve. +func TestInvariant_Links(t *testing.T) { + t.Run("resolving link passes", func(t *testing.T) { + ctxDir := t.TempDir() + writeThemeFile(t, filepath.Join(ctxDir, "learnings"), "hooks.md", "# hooks\n") + root := themeWith("hooks", "learnings/hooks.md") + if err := disclosure.CheckLinks(root, ctxDir); err != nil { + t.Errorf("CheckLinks = %v, want nil", err) + } + }) + + t.Run("broken link", func(t *testing.T) { + ctxDir := t.TempDir() // nothing created + root := themeWith("hooks", "learnings/hooks.md") + if err := disclosure.CheckLinks(root, ctxDir); !errors.Is(err, errDisc.ErrBrokenThemeLink) { + t.Errorf("CheckLinks = %v, want ErrBrokenThemeLink", err) + } + }) +} diff --git a/internal/disclosure/kind.go b/internal/disclosure/kind.go new file mode 100644 index 000000000..6e4c738b0 --- /dev/null +++ b/internal/disclosure/kind.go @@ -0,0 +1,54 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + cfgCtx "github.com/ActiveMemory/ctx/internal/config/ctx" + cfgEntry "github.com/ActiveMemory/ctx/internal/config/entry" +) + +// KindFor maps a canonical knowledge-file basename to its Kind. It is +// how the CLI decides which root it was handed; a non-knowledge file +// returns false so the caller can refuse rather than guess. +// +// Parameters: +// - basename: a file's base name (e.g. "LEARNINGS.md") +// +// Returns: +// - Kind: the matched kind (meaningful only when ok is true) +// - bool: true when basename is a canonical knowledge file +func KindFor(basename string) (Kind, bool) { + switch basename { + case cfgCtx.Learning: + return KindLearning, true + case cfgCtx.Decision: + return KindDecision, true + case cfgCtx.Convention: + return KindConvention, true + default: + return KindLearning, false + } +} + +// String returns the kind's name, matching the entry-type vocabulary +// ("learning" | "decision" | "convention"). Used for the Inspection's +// stable string Kind field. +// +// Returns: +// - string: the kind name, or "" for an unknown kind +func (k Kind) String() string { + switch k { + case KindLearning: + return cfgEntry.Learning + case KindDecision: + return cfgEntry.Decision + case KindConvention: + return cfgEntry.Convention + default: + return "" + } +} diff --git a/internal/disclosure/parse.go b/internal/disclosure/parse.go new file mode 100644 index 000000000..fe9003355 --- /dev/null +++ b/internal/disclosure/parse.go @@ -0,0 +1,56 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + cfgDisc "github.com/ActiveMemory/ctx/internal/config/disclosure" +) + +// Parse splits a root's content into its regions (preamble, staging, +// themes) for the given kind. It is total: any byte string splits into +// segments, so parsing never fails — all refusal lives in Validate. +// +// The raw segments are kept verbatim, so Reconstruct returns the input +// byte-for-byte; nothing is normalized. Themes is the parsed view of the +// raw themes region. HasThemes is false for a not-yet-migrated root. +// +// Layout per kind (see specs/progressive-disclosure.md): +// - entry kinds: preamble | staging (## [ entries) | ## Themes … +// - conventions: preamble | ## Themes … | ## Recent (staging) +// +// Parameters: +// - content: the full root file content +// - k: which canonical file this is +// +// Returns: +// - Root: the split root; Reconstruct(r) == content +func Parse(content string, k Kind) Root { + r := Root{Kind: k} + themeOffsets := headingLineOffsets(content, cfgDisc.HeadingThemes) + r.HasThemes = len(themeOffsets) > 0 + + if k == KindConvention { + parseConvention(&r, content, themeOffsets) + } else { + parseEntryKind(&r, content, themeOffsets) + } + + r.Themes = parseThemes(r.ThemesRaw) + return r +} + +// Reconstruct returns the root's content in file order for its kind. It +// is the inverse of Parse: Reconstruct(Parse(c, k)) == c. +// +// Returns: +// - string: the reassembled root content +func (r Root) Reconstruct() string { + if r.Kind == KindConvention { + return r.Preamble + r.ThemesRaw + r.Staging + } + return r.Preamble + r.Staging + r.ThemesRaw +} diff --git a/internal/disclosure/parse_test.go b/internal/disclosure/parse_test.go new file mode 100644 index 000000000..69e805c9b --- /dev/null +++ b/internal/disclosure/parse_test.go @@ -0,0 +1,111 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure_test + +import ( + "strings" + "testing" + + "github.com/ActiveMemory/ctx/internal/disclosure" +) + +// Entry-kind fixtures (LEARNINGS/DECISIONS): staging above ## Themes. +const ( + entryMigratedPopulated = "# Learnings\n\n<!-- guide -->\n\n" + + "## [2026-07-15-120000] a staged entry\n\n**Context**: x.\n\n---\n\n" + + "## Themes\n\n- hooks — hook mechanics → [hooks](learnings/hooks.md)\n" + + entryMigratedEmpty = "# Learnings\n\n<!-- guide -->\n\n" + + "## Themes\n\n- hooks — hook mechanics → [hooks](learnings/hooks.md)\n" + + entryUnmigrated = "# Learnings\n\n<!-- guide -->\n\n" + + "## [2026-07-15-120000] a staged entry\n\n**Context**: x.\n" + + conventionMigrated = "# Conventions\n\n<!-- guide -->\n\n" + + "## Themes\n\n- naming — file naming → [naming](conventions/naming.md)\n\n" + + "## Recent\n\n### a recent convention\n\nprose.\n" + + conventionUnmigrated = "# Conventions\n\n<!-- guide -->\n\n" + + "### a convention\n\nprose.\n" +) + +// T04/T05: Parse must round-trip every shape byte-for-byte — nothing is +// normalized, so the mover (M2) gets exact bytes. +func TestParse_RoundTrip(t *testing.T) { + cases := []struct { + name string + content string + kind disclosure.Kind + }{ + {"entry migrated populated", entryMigratedPopulated, disclosure.KindLearning}, + {"entry migrated empty staging", entryMigratedEmpty, disclosure.KindLearning}, + {"entry un-migrated", entryUnmigrated, disclosure.KindDecision}, + {"convention migrated", conventionMigrated, disclosure.KindConvention}, + {"convention un-migrated", conventionUnmigrated, disclosure.KindConvention}, + {"empty", "", disclosure.KindLearning}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := disclosure.Parse(tc.content, tc.kind).Reconstruct() + if got != tc.content { + t.Errorf("round-trip mismatch\n got: %q\nwant: %q", got, tc.content) + } + }) + } +} + +// T04: entry-kind split places entries in staging and gists in themes. +func TestParse_EntryKind(t *testing.T) { + r := disclosure.Parse(entryMigratedPopulated, disclosure.KindLearning) + if !r.HasThemes { + t.Fatal("HasThemes = false, want true (## Themes present)") + } + if !strings.Contains(r.Staging, "a staged entry") { + t.Errorf("staging missing the entry; staging=%q", r.Staging) + } + if strings.Contains(r.Staging, "## Themes") { + t.Errorf("staging leaked the themes section; staging=%q", r.Staging) + } + if len(r.Themes) != 1 || r.Themes[0].Link != "learnings/hooks.md" { + t.Errorf("themes parse wrong: %+v", r.Themes) + } +} + +// T04: an un-migrated entry root has empty themes and all entries staged. +func TestParse_EntryKindUnmigrated(t *testing.T) { + r := disclosure.Parse(entryUnmigrated, disclosure.KindLearning) + if r.HasThemes { + t.Error("HasThemes = true, want false (no ## Themes)") + } + if !strings.Contains(r.Staging, "a staged entry") { + t.Errorf("entry not in staging; staging=%q", r.Staging) + } + if r.ThemesRaw != "" { + t.Errorf("ThemesRaw = %q, want empty", r.ThemesRaw) + } +} + +// T05: conventions split has themes between ## Themes and ## Recent, and +// staging is the ## Recent region. +func TestParse_ConventionKind(t *testing.T) { + r := disclosure.Parse(conventionMigrated, disclosure.KindConvention) + if !r.HasThemes { + t.Fatal("HasThemes = false, want true") + } + if !strings.HasPrefix(strings.TrimSpace(r.Staging), "## Recent") { + t.Errorf("staging must start at ## Recent; staging=%q", r.Staging) + } + if !strings.Contains(r.Staging, "a recent convention") { + t.Errorf("recent section not in staging; staging=%q", r.Staging) + } + if strings.Contains(r.ThemesRaw, "## Recent") { + t.Errorf("themes region leaked ## Recent; themesRaw=%q", r.ThemesRaw) + } + if len(r.Themes) != 1 || r.Themes[0].Link != "conventions/naming.md" { + t.Errorf("themes parse wrong: %+v", r.Themes) + } +} diff --git a/internal/disclosure/regions.go b/internal/disclosure/regions.go new file mode 100644 index 000000000..bb45ac944 --- /dev/null +++ b/internal/disclosure/regions.go @@ -0,0 +1,189 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + "strings" + + cfgDisc "github.com/ActiveMemory/ctx/internal/config/disclosure" + "github.com/ActiveMemory/ctx/internal/config/token" +) + +// parseEntryKind splits an entry-based root: preamble, then the staging +// entries, then the ## Themes region (which runs to EOF). It writes the +// Preamble, Staging, and ThemesRaw fields of r. +// +// Parameters: +// - r: the root being assembled (mutated in place) +// - content: the full root content +// - themeOffsets: byte offsets of every ## Themes heading line +func parseEntryKind(r *Root, content string, themeOffsets []int) { + beforeThemes := content + if r.HasThemes { + t := themeOffsets[0] + beforeThemes = content[:t] + r.ThemesRaw = content[t:] + } + + si := firstLinePrefixOffset(beforeThemes, cfgDisc.EntryLinePrefix) + if si != -1 { + r.Preamble = beforeThemes[:si] + r.Staging = beforeThemes[si:] + } else { + r.Preamble = beforeThemes + } +} + +// parseConvention splits a conventions root: preamble, the ## Themes +// region, then the ## Recent staging region. An un-migrated conventions +// file has no ## Themes; its prose sections are the staging directly. +// +// Parameters: +// - r: the root being assembled (mutated in place) +// - content: the full root content +// - themeOffsets: byte offsets of every ## Themes heading line +func parseConvention(r *Root, content string, themeOffsets []int) { + if !r.HasThemes { + si := firstLinePrefixOffset(content, cfgDisc.ConventionLinePrefix) + if si != -1 { + r.Preamble = content[:si] + r.Staging = content[si:] + } else { + r.Preamble = content + } + return + } + + t := themeOffsets[0] + r.Preamble = content[:t] + recentOffsets := headingLineOffsets(content[t:], cfgDisc.HeadingRecent) + if len(recentOffsets) > 0 { + rec := t + recentOffsets[0] + r.ThemesRaw = content[t:rec] + r.Staging = content[rec:] + } else { + r.ThemesRaw = content[t:] + } +} + +// parseThemes reads the "- name — gist → [label](link)" bullet lines of a +// raw ## Themes region into Theme values. It is best-effort: lines it +// does not recognize are skipped (Validate, not Parse, judges shape). +// +// Parameters: +// - themesRaw: the raw ## Themes region of a root +// +// Returns: +// - []Theme: one per recognized bullet, in file order; nil if none +func parseThemes(themesRaw string) []Theme { + if themesRaw == "" { + return nil + } + var themes []Theme + for _, line := range strings.Split(themesRaw, token.NewlineLF) { + trimmed := strings.TrimSpace(line) + if !strings.HasPrefix(trimmed, token.PrefixListDash) { + continue + } + body := strings.TrimPrefix(trimmed, token.PrefixListDash) + themes = append(themes, parseThemeBullet(body)) + } + return themes +} + +// parseThemeBullet pulls the name, gist, and link out of one theme +// bullet's body. The link is the first "(...)" following a "](". +// +// Parameters: +// - body: one theme bullet with the leading "- " already stripped +// +// Returns: +// - Theme: the parsed name/gist/link (fields empty when not present) +func parseThemeBullet(body string) Theme { + t := Theme{} + if open := strings.LastIndex(body, cfgDisc.LinkOpen); open != -1 { + start := open + len(cfgDisc.LinkOpen) + if end := strings.IndexByte(body[start:], ')'); end != -1 { + t.Link = body[start : start+end] + } + } + // name is the text before the em-dash metadata separator. + if dash := strings.Index(body, token.MetaSeparator); dash != -1 { + t.Name = strings.TrimSpace(body[:dash]) + t.Gist = strings.TrimSpace(body[dash+len(token.MetaSeparator):]) + } else { + t.Name = strings.TrimSpace(body) + } + return t +} + +// headingLineOffsets returns the byte offset of every line whose trimmed +// content equals heading (an exact ATX heading line). Used to find and +// count region-delimiting headings. +// +// Parameters: +// - content: the text to scan +// - heading: the exact heading line to match (e.g. "## Themes") +// +// Returns: +// - []int: byte offsets of each matching line's start, in order +func headingLineOffsets(content, heading string) []int { + var offsets []int + for i := 0; i < len(content); { + line, next := lineAt(content, i) + if strings.TrimSpace(line) == heading { + offsets = append(offsets, i) + } + if next == -1 { + break + } + i = next + } + return offsets +} + +// firstLinePrefixOffset returns the byte offset of the first line that +// starts with prefix, or -1. +// +// Parameters: +// - content: the text to scan +// - prefix: the line-start prefix to match (e.g. "## [") +// +// Returns: +// - int: byte offset of the first matching line, or -1 if none +func firstLinePrefixOffset(content, prefix string) int { + for i := 0; i < len(content); { + line, next := lineAt(content, i) + if strings.HasPrefix(line, prefix) { + return i + } + if next == -1 { + break + } + i = next + } + return -1 +} + +// lineAt returns the line beginning at byte offset i (without its +// trailing newline) and the offset of the next line's start, or -1 when +// this is the last line. +// +// Parameters: +// - content: the text being scanned +// - i: byte offset of the start of the line to read +// +// Returns: +// - line: the line's content without its trailing newline +// - next: byte offset of the next line's start, or -1 if last +func lineAt(content string, i int) (line string, next int) { + rel := strings.Index(content[i:], token.NewlineLF) + if rel == -1 { + return content[i:], -1 + } + return content[i : i+rel], i + rel + 1 +} diff --git a/internal/disclosure/testmain_test.go b/internal/disclosure/testmain_test.go new file mode 100644 index 000000000..92eaa3f4b --- /dev/null +++ b/internal/disclosure/testmain_test.go @@ -0,0 +1,22 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure_test + +import ( + "os" + "testing" + + "github.com/ActiveMemory/ctx/internal/assets/read/lookup" +) + +// TestMain initializes the embedded asset lookup so sentinel .Error() +// text resolves — otherwise a failing assertion prints an empty error. +// (See the "uninitialized desc.Text" learning.) +func TestMain(m *testing.M) { + lookup.Init() + os.Exit(m.Run()) +} diff --git a/internal/disclosure/types.go b/internal/disclosure/types.go new file mode 100644 index 000000000..83ca00141 --- /dev/null +++ b/internal/disclosure/types.go @@ -0,0 +1,90 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +// Kind identifies which canonical knowledge file a root is, because the +// staging zone sits in a different place per kind (above ## Themes for +// entry files, inside ## Recent for conventions). +type Kind int + +const ( + // KindLearning is LEARNINGS.md — "## [ts] Title" entries, staging + // above ## Themes. + KindLearning Kind = iota + // KindDecision is DECISIONS.md — same entry shape as learnings. + KindDecision + // KindConvention is CONVENTIONS.md — prose sections, staging in + // ## Recent. + KindConvention +) + +// Theme is one entry in a root's ## Themes section: a name, the +// "just enough" gist, and the markdown link to the theme file holding +// that theme's bodies. +// +// Fields: +// - Name: the theme's short name (heading text) +// - Gist: the "just enough" one-liner conveying the theme +// - Link: relative path to the theme file holding the bodies +type Theme struct { + Name string `json:"name"` + Gist string `json:"gist"` + Link string `json:"link"` +} + +// Root is a parsed progressive-disclosure root, split into its regions. +// +// The raw segments (Preamble, Staging, ThemesRaw) are kept verbatim so a +// root round-trips byte-for-byte via Reconstruct: parsing never +// normalizes content, which is how the clobber-bug class is avoided. +// Themes is the parsed view of ThemesRaw, for consumers that read the +// gists. +// +// HasThemes is false for a not-yet-migrated root (no ## Themes section); +// Validate treats that as the first-run case, not an error. +// +// Fields: +// - Kind: which canonical file this root is +// - Preamble: everything above the first region (H1 + comment block) +// - Staging: the un-digested entries region (verbatim) +// - ThemesRaw: the raw ## Themes region (verbatim), empty if none +// - Themes: parsed view of ThemesRaw +// - HasThemes: false when the root is not yet migrated +type Root struct { + Kind Kind + Preamble string + Staging string + ThemesRaw string + Themes []Theme + HasThemes bool +} + +// StagedEntry is one un-digested entry in a root's staging zone, +// identified for the dry-run pass to propose a theme for. +// +// Fields: +// - Timestamp: the entry's timestamp (e.g. "2026-07-18-120000") +// - Title: the entry's title text +type StagedEntry struct { + Timestamp string `json:"timestamp"` + Title string `json:"title"` +} + +// Inspection is the read-only view of a root the dry-run pass consumes: +// what kind it is, which entries are staged (awaiting digestion), and +// which themes already exist. It is the structured form of a Parse, +// stable across the CLI's JSON output. +// +// Fields: +// - Kind: the root's kind name ("learning" | "decision" | "convention") +// - Staging: the un-digested entries, in file order +// - Themes: the current ## Themes (name/gist/link), in file order +type Inspection struct { + Kind string `json:"kind"` + Staging []StagedEntry `json:"staging"` + Themes []Theme `json:"themes"` +} diff --git a/internal/disclosure/validate.go b/internal/disclosure/validate.go new file mode 100644 index 000000000..2bf4e841b --- /dev/null +++ b/internal/disclosure/validate.go @@ -0,0 +1,56 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + "strings" + + cfgDisc "github.com/ActiveMemory/ctx/internal/config/disclosure" + errDisc "github.com/ActiveMemory/ctx/internal/err/disclosure" + "github.com/ActiveMemory/ctx/internal/heading" +) + +// Validate is the progressive-disclosure precondition (spec Guards §2). +// It refuses a structurally malformed root, so the pass never mutates +// content it does not understand — the failure mode behind the clobber +// bug. It is fail-loud with no auto-repair. +// +// Rules: +// - zero or one "## Themes". Zero is the not-yet-migrated first-run +// case and is valid; two or more is malformed (ErrMultipleThemes). +// - no "## [" entry below "## Themes" (ErrEntryBelowThemes): entries +// must stay in the staging zone above it. +// - for entry kinds, a non-empty staging zone must parse into discrete +// "## [" entries (ErrStagingUnparsable). +// +// Invariants that are vacuously true on an un-migrated root (no themes, +// no theme files) need no special-casing here. +// +// Parameters: +// - r: a parsed root (from Parse) +// +// Returns: +// - error: one of the disclosure sentinels, or nil when well-formed +func Validate(r Root) error { + if len(headingLineOffsets(r.Reconstruct(), cfgDisc.HeadingThemes)) > 1 { + return errDisc.ErrMultipleThemes + } + + entryBelow := r.HasThemes && + firstLinePrefixOffset(r.ThemesRaw, cfgDisc.EntryLinePrefix) != -1 + if entryBelow { + return errDisc.ErrEntryBelowThemes + } + + if r.Kind != KindConvention && + strings.TrimSpace(r.Staging) != "" && + len(heading.ParseEntryBlocks(r.Staging)) == 0 { + return errDisc.ErrStagingUnparsable + } + + return nil +} diff --git a/internal/disclosure/validate_test.go b/internal/disclosure/validate_test.go new file mode 100644 index 000000000..05c419a78 --- /dev/null +++ b/internal/disclosure/validate_test.go @@ -0,0 +1,58 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure_test + +import ( + "errors" + "testing" + + "github.com/ActiveMemory/ctx/internal/disclosure" + errDisc "github.com/ActiveMemory/ctx/internal/err/disclosure" +) + +const ( + twoThemes = "# Learnings\n\n## Themes\n\n- a — g → [a](learnings/a.md)\n\n" + + "## Themes\n\n- b — g → [b](learnings/b.md)\n" + + entryBelowThemes = "# Learnings\n\n## Themes\n\n- a — g → [a](learnings/a.md)\n\n" + + "## [2026-07-15-120000] misplaced entry\n\n**Context**: below themes.\n" + + unparsableStaging = "# Learnings\n\n<!-- guide -->\n\n" + + "## [not-a-real-timestamp] malformed\n\nbody.\n\n" + + "## Themes\n\n- a — g → [a](learnings/a.md)\n" +) + +// T06: the Validate precondition returns the named sentinel for each +// malformed shape, and nil for the two valid shapes (well-formed and +// not-yet-migrated). +func TestValidate(t *testing.T) { + cases := []struct { + name string + content string + kind disclosure.Kind + want error // nil = must pass + }{ + {"well-formed populated", entryMigratedPopulated, disclosure.KindLearning, nil}, + {"un-migrated (zero themes)", entryUnmigrated, disclosure.KindLearning, nil}, + {"well-formed empty staging", entryMigratedEmpty, disclosure.KindLearning, nil}, + {"convention well-formed", conventionMigrated, disclosure.KindConvention, nil}, + {"two ## Themes", twoThemes, disclosure.KindLearning, errDisc.ErrMultipleThemes}, + {"entry below themes", entryBelowThemes, disclosure.KindLearning, errDisc.ErrEntryBelowThemes}, + {"unparsable staging", unparsableStaging, disclosure.KindLearning, errDisc.ErrStagingUnparsable}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := disclosure.Validate(disclosure.Parse(tc.content, tc.kind)) + switch { + case tc.want == nil && got != nil: + t.Errorf("Validate = %v, want nil (valid shape)", got) + case tc.want != nil && !errors.Is(got, tc.want): + t.Errorf("Validate = %v, want %v", got, tc.want) + } + }) + } +} diff --git a/internal/err/disclosure/disclosure.go b/internal/err/disclosure/disclosure.go new file mode 100644 index 000000000..fd2e56fac --- /dev/null +++ b/internal/err/disclosure/disclosure.go @@ -0,0 +1,83 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + "fmt" + + "github.com/ActiveMemory/ctx/internal/assets/read/desc" + "github.com/ActiveMemory/ctx/internal/config/embed/text" + "github.com/ActiveMemory/ctx/internal/entity" +) + +const ( + // ErrMultipleThemes: the root has more than one "## Themes" section. + // Validate refuses rather than guess which is authoritative. + ErrMultipleThemes = entity.Sentinel( + text.DescKeyErrDisclosureMultipleThemes, + ) + + // ErrEntryBelowThemes: a "## [" entry sits below "## Themes". Entries + // must stay in the staging zone above it, or the pass cannot find them. + ErrEntryBelowThemes = entity.Sentinel( + text.DescKeyErrDisclosureEntryBelowThemes, + ) + + // ErrStagingUnparsable: the staging zone did not parse into discrete + // entries. Refuse rather than regenerate from "what was recognized". + ErrStagingUnparsable = entity.Sentinel( + text.DescKeyErrDisclosureStagingUnparsable, + ) + + // ErrOrphanThemeFile: a theme file exists with no matching gist in + // the root — the root no longer reaches it (link-graph orphan). + ErrOrphanThemeFile = entity.Sentinel( + text.DescKeyErrDisclosureOrphanThemeFile, + ) + + // ErrMissingThemeFile: a root gist links to a theme file that does + // not exist. + ErrMissingThemeFile = entity.Sentinel( + text.DescKeyErrDisclosureMissingThemeFile, + ) + + // ErrDuplicateEntry: an entry appears in more than one place (staging + // and a theme file, or two theme files) — the "exactly one place" + // invariant is broken. + ErrDuplicateEntry = entity.Sentinel( + text.DescKeyErrDisclosureDuplicateEntry, + ) + + // ErrBrokenThemeLink: a theme link in the root points at a path that + // does not resolve on disk. + ErrBrokenThemeLink = entity.Sentinel( + text.DescKeyErrDisclosureBrokenThemeLink, + ) + + // ErrNotAKnowledgeFile: the file handed to `ctx disclosure` is not a + // canonical knowledge file (LEARNINGS/DECISIONS/CONVENTIONS). Wrap + // with [NotAKnowledgeFile] to name the offending path. + ErrNotAKnowledgeFile = entity.Sentinel( + text.DescKeyErrDisclosureNotKnowledgeFileMsg, + ) +) + +// NotAKnowledgeFile wraps [ErrNotAKnowledgeFile] with the offending path +// and the expected filenames. +// +// Parameters: +// - path: the file that was not a canonical knowledge file +// +// Returns: +// - error: wrapping ErrNotAKnowledgeFile for errors.Is matches +func NotAKnowledgeFile(path string) error { + return fmt.Errorf( + desc.Text(text.DescKeyErrDisclosureNotKnowledgeFile), + ErrNotAKnowledgeFile, + path, + ) +} diff --git a/internal/err/disclosure/doc.go b/internal/err/disclosure/doc.go new file mode 100644 index 000000000..6a85a4111 --- /dev/null +++ b/internal/err/disclosure/doc.go @@ -0,0 +1,37 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +// Package disclosure holds the identity sentinels for the +// progressive-disclosure guards and invariants (see +// specs/progressive-disclosure.md). +// +// # Domain +// +// Each sentinel is an [github.com/ActiveMemory/ctx/internal/entity.Sentinel] +// — a string const whose user-facing text is resolved from +// commands/text/errors.yaml at call time. Callers match them with +// errors.Is against the value returned by [Validate] and the invariant +// checks in internal/disclosure. +// +// They fall into two groups: +// +// - **Structure** ([ErrMultipleThemes], [ErrEntryBelowThemes], +// [ErrStagingUnparsable]): the precondition refused a malformed root. +// - **Cross-file** ([ErrOrphanThemeFile], [ErrMissingThemeFile], +// [ErrDuplicateEntry], [ErrBrokenThemeLink]): the root ↔ theme-file +// link graph or the one-place-per-entry invariant is broken. +// +// # Wrapping strategy +// +// Milestone 1 returns these bare — the guard identity is what the checks +// assert. Parameterized wrappers that name the specific theme file or +// entry are deferred to the digesting pass (a later milestone), which is +// where the message reaches an operator. +// +// # Concurrency +// +// Package-level constants; safe for concurrent use. +package disclosure diff --git a/internal/write/disclosure/disclosure.go b/internal/write/disclosure/disclosure.go new file mode 100644 index 000000000..c5bf4c300 --- /dev/null +++ b/internal/write/disclosure/disclosure.go @@ -0,0 +1,71 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +package disclosure + +import ( + "encoding/json" + "fmt" + + "github.com/spf13/cobra" + + "github.com/ActiveMemory/ctx/internal/assets/read/desc" + "github.com/ActiveMemory/ctx/internal/config/embed/text" + "github.com/ActiveMemory/ctx/internal/config/token" + "github.com/ActiveMemory/ctx/internal/disclosure" +) + +// Human prints an Inspection as a scannable summary: the kind, the +// staged entries awaiting digestion, and the current themes. +// +// Parameters: +// - cmd: Cobra command for the output stream +// - insp: the inspection to render +func Human(cmd *cobra.Command, insp disclosure.Inspection) { + cmd.Println(fmt.Sprintf(desc.Text(text.DescKeyWriteDisclosureKind), insp.Kind)) + + cmd.Println(fmt.Sprintf( + desc.Text(text.DescKeyWriteDisclosureStagedHeader), len(insp.Staging), + )) + if len(insp.Staging) == 0 { + cmd.Println(desc.Text(text.DescKeyWriteDisclosureNone)) + } + for _, e := range insp.Staging { + cmd.Println(fmt.Sprintf( + desc.Text(text.DescKeyWriteDisclosureStagedLine), + e.Timestamp, e.Title, + )) + } + + cmd.Println(fmt.Sprintf( + desc.Text(text.DescKeyWriteDisclosureThemesHeader), len(insp.Themes), + )) + if len(insp.Themes) == 0 { + cmd.Println(desc.Text(text.DescKeyWriteDisclosureNone)) + } + for _, t := range insp.Themes { + cmd.Println(fmt.Sprintf( + desc.Text(text.DescKeyWriteDisclosureThemeLine), t.Name, t.Link, + )) + } +} + +// JSON prints an Inspection as indented JSON for machine consumption. +// +// Parameters: +// - cmd: Cobra command for the output stream +// - insp: the inspection to render +// +// Returns: +// - error: non-nil only if JSON marshaling fails +func JSON(cmd *cobra.Command, insp disclosure.Inspection) error { + b, err := json.MarshalIndent(insp, "", token.Space+token.Space) + if err != nil { + return err + } + cmd.Println(string(b)) + return nil +} diff --git a/internal/write/disclosure/doc.go b/internal/write/disclosure/doc.go new file mode 100644 index 000000000..1b567e31e --- /dev/null +++ b/internal/write/disclosure/doc.go @@ -0,0 +1,21 @@ +// / ctx: https://ctx.ist +// ,'`./ do you remember? +// `.,'\ +// \ Copyright 2026-present Context contributors. +// SPDX-License-Identifier: Apache-2.0 + +// Package disclosure renders the output of the `ctx disclosure` command +// family: a knowledge root's staged entries and current themes, as a +// human summary ([Human]) or machine JSON ([JSON]). +// +// # Domain +// +// It consumes the read-only Inspection value from internal/disclosure +// and writes to the cobra command's output stream. All labels are +// resolved through internal/assets/read/desc from +// commands/text/write.yaml — no English literals live here. +// +// # Concurrency +// +// Stateless formatting; safe for concurrent use given distinct writers. +package disclosure diff --git a/specs/fix-afterheader-tail-truncation.md b/specs/fix-afterheader-tail-truncation.md new file mode 100644 index 000000000..647aa6a51 --- /dev/null +++ b/specs/fix-afterheader-tail-truncation.md @@ -0,0 +1,94 @@ +# Spec: fix AfterHeader dropping everything after the insert point + +## Problem + +`insert.AfterHeader` (`internal/cli/add/core/insert/insert.go`) ends with: + +```go +return []byte(content[:insertPoint] + entry) +``` + +It truncates the file at `insertPoint` and appends the entry — the tail +(`content[insertPoint:]`) is **silently discarded**. It is an insert +function that does not insert. + +Its sole caller is `beforeFirstEntry`'s fallback +(`internal/cli/add/core/insert/before.go`), taken when a knowledge file +contains no `## [` entry. So `ctx learning add` / `ctx decision add` on +an entry-less LEARNINGS.md/DECISIONS.md destroys any content that sits +below the H1 header and its comment block. + +Today this is **masked**, not absent: an entry-less file freshly created +by `ctx init` has nothing after the comment block, so `insertPoint` +lands at EOF and the discarded tail is empty. The bug bites the moment +non-entry content exists below the preamble of a file that has no +entries yet — e.g. a hand-written `## Notes` section, or any structural +section a future feature adds. + +This is the same family as the `ctx learning add` clobber bug that +`index.Validate` exists to guard: silent destruction of persisted +memory, recoverable only from git. + +Discovered by the layout proof written for +`specs/progressive-disclosure.md` (plan `pd-m1`, T10), which planted a +`## Themes` section below the preamble of an entry-less root and watched +`add` delete it. The fix is specified and shipped **independently** of +that design: it is a defect in current behavior on its own terms. + +## Design + +Make `AfterHeader` a true insert, preserving the tail — the pattern its +sibling `Task` already follows +(`existingStr[:pendingIdx] + entry + NewlineLF + existingStr[pendingIdx:]`). + +When the tail is non-empty, separate the new entry from the following +content with the same delimiter `beforeFirstEntry`'s primary path uses, +so both paths in the same function family produce consistent files: + +``` +entry + NewlineLF + Separator + NewlineLF + NewlineLF + tail +``` + +When the tail is **empty**, emit `content[:insertPoint] + entry` +unchanged — byte-identical to today's output. This keeps the fix a +strict superset of current behavior: every file shape that works today +produces identical bytes, and only the shape that currently loses data +changes. + +## Implementation + +- `internal/cli/add/core/insert/insert.go`: `AfterHeader` returns + `content[:insertPoint] + entry` when `content[insertPoint:]` is empty, + and `content[:insertPoint] + entry + sep + tail` otherwise. +- No signature change; one caller; no other call sites. + +## Tests + +The package had **no tests**. Add: + +- `testmain_test.go`: `lookup.Init()` in `TestMain`. Without it + `desc.Text` returns `""` and `strings.Index(s, "")` is `0`, which + silently turns every insert anchor into "match at offset 0" — tests + would exercise a path production never takes and pass for the wrong + reason. This is a precondition for any honest test of this package. +- Tail preserved: content below the preamble of an entry-less file + survives an add, and the entry lands above it. +- Empty tail unchanged: an entry-less file with nothing after the + comment block produces byte-identical output to the pre-fix + implementation. +- Primary path untouched: a file with `## [` entries still inserts + before the first one (regression guard for `beforeFirstEntry`). + +## Acceptance + +- `ctx learning add` against an entry-less file carrying a section below + the preamble preserves that section. +- Output for every file shape that works today is byte-identical. +- `make lint` clean; full suite green. + +## Non-Goals + +- No change to `beforeFirstEntry`'s primary (`## [`-anchored) path. +- No change to `Task`, `TaskAfterSection`, `AppendAtEnd`, or `Decision`. +- Not a progressive-disclosure change: that design merely *found* this; + the fix stands alone and ships alone. diff --git a/specs/plans/pd-m1.md b/specs/plans/pd-m1.md new file mode 100644 index 000000000..b4bba0376 --- /dev/null +++ b/specs/plans/pd-m1.md @@ -0,0 +1,175 @@ +# pd-m1 Plan — guards, invariants, and structural vocabulary + +**Spec:** `specs/progressive-disclosure.md` · **Status:** Ready +**Blocking TBDs resolved:** B1 (validate vs first-run contradiction) — +resolved **in the spec**, Guards §2: validate accepts **zero or one** +`## Themes`; zero = not-yet-migrated (the pass creates it), two+ = +malformed → refuse. Invariants need no carve-out (vacuously true on an +un-migrated root). + +## Scope & DoD + +Milestone 1 builds the **guards, invariants, and structural vocabulary** +and proves the layout premise. **Nothing moves**: no entry body is +relocated, no gist is authored, no `.context` knowledge content changes. +This ordering is deliberate — the pass (M2+) moves entry bodies, which is +the clobber/data-loss risk class, so the refusal machinery and the +layout proof land first. + +DoD (confirmed by measurement or by the user — never derived from task +completion): + +- [x] `make lint` reports 0 issues and `go test ./...` is green +- [x] The **layout proof** passes for all three kinds, with populated + *and* empty staging: `add` lands in the staging zone with a + `## Themes` section present (T10–T12) +- [x] `Validate` refuses every malformed shape in the test matrix with + the correct sentinel (T06) +- [x] LEARNINGS.md / DECISIONS.md / CONVENTIONS.md bodies are + **byte-identical** to milestone start (`git diff --stat` shows no + change to them) + +## Data model & storage + +No persistence, no migrations, no DDL. In-memory only: + +```go +type Kind int // KindLearning | KindDecision | KindConvention + +type Theme struct { + Name string // heading text under ## Themes + Gist string // the "just enough" line(s) + Link string // relative path to the theme file +} + +type Root struct { + Preamble string // everything before the staging zone + Staging string // raw staging region (entry bodies live here) + Themes []Theme // parsed ## Themes section (empty when un-migrated) + HasThemes bool // false = not yet migrated (first run) +} +``` + +`Staging` stays **raw** in M1: the mover (M2) needs byte-exact bodies, +and re-serializing parsed entries is how content gets silently +normalized. Parsing into blocks uses the existing +`heading.ParseEntryBlocks`; M1 never writes a root back. + +## Contracts + +New package `internal/disclosure`: + +- `Parse(content string, k Kind) (Root, error)` — splits a root into + preamble / staging / themes. Round-trips: `Preamble+Staging+Themes` + reconstructs the input byte-for-byte. +- `Validate(r Root) error` — the precondition (spec Guards §2). +- `CheckPairing(root Root, themeDir string) error` — gists ↔ theme files 1:1. +- `CheckUniqueness(root Root, themeDir string) error` — entry in exactly one place. +- `CheckLinks(root Root, ctxDir string) error` — every theme link resolves. + +Constants → `internal/config/disclosure`: `HeadingThemes = "## Themes"`, +`HeadingRecent = "## Recent"`. +Errors → `internal/err/disclosure` (constructors in `internal/err` per +CONVENTIONS; sentinels are `entity.Sentinel` consts with text in +`commands/text/errors.yaml` keyed `err.disclosure.*` — never English in +Go). + +No CLI surface in M1. No `ctx agent` change (spec Non-Goals). + +## Test matrix + +| Invariant / rule | Violation attempt | Expected failure | Task | +|---|---|---|---| +| zero-or-one `## Themes` | two `## Themes` headings | `Validate` → `ErrMultipleThemes` | T06 | +| un-migrated root is valid | zero `## Themes` | `Validate` → `nil` (first run) | T06 | +| no `## [` below `## Themes` | entry planted below Themes | `Validate` → `ErrEntryBelowThemes` | T06 | +| staging parses into discrete entries | malformed staging region | `Validate` → `ErrStagingUnparsable` | T06 | +| gists ↔ theme files 1:1 | theme file with no gist | `CheckPairing` → `ErrOrphanThemeFile` | T07 | +| gists ↔ theme files 1:1 | gist with no theme file | `CheckPairing` → `ErrMissingThemeFile` | T07 | +| pairing vacuous when un-migrated | 0 gists, 0 theme files | `CheckPairing` → `nil` | T07 | +| entry in exactly one place | same entry in staging *and* a theme file | `CheckUniqueness` → `ErrDuplicateEntry` | T08 | +| entry in exactly one place | same entry in two theme files | `CheckUniqueness` → `ErrDuplicateEntry` | T08 | +| theme links resolve | link to nonexistent path | `CheckLinks` → `ErrBrokenThemeLink` | T09 | +| add lands in staging (learning) | Themes present, staging populated | new entry index `<` Themes index | T10 | +| add lands in staging (learning) | Themes present, **staging empty** | new entry index `<` Themes index (AfterHeader fallback) | T10 | +| add lands in staging (decision) | both above cases | new entry index `<` Themes index | T11 | +| add lands in staging (convention) | Themes present | new section lands inside `## Recent`, after Themes | T12 | +| invariants tolerate the real tree | run against today's `.context` | vacuous pass | T14 | + +## Task breakdown + +| id | st | task | deps | files | [P] | acceptance criterion | spec ref | +|---|---|---|---|---|---|---|---| +| T01 | [x] | Structural vocabulary constants | — | `internal/config/disclosure/disclosure.go` | [P] | `go test ./internal/config/disclosure/` asserts `HeadingThemes == "## Themes"` and `HeadingRecent == "## Recent"` | Design/Layout | +| T02 | [x] | Error sentinels + i18n text | — | `internal/err/disclosure/*.go`, `commands/text/errors.yaml` | [P] | `go test ./internal/err/...` green; test asserts every `err.disclosure.*` sentinel's `Error()` resolves non-empty via `desc.Text` (no English literal in Go) | CONVENTIONS/Error Handling | +| T03 | [x] | `Root`/`Theme`/`Kind` types | — | `internal/disclosure/types.go` | [P] | `go build ./...` green; `make audit` type-file report shows 0 violations | Data model | +| T04 | [x] | `Parse` for entry kinds (LEARNINGS/DECISIONS) | T01,T03 | `internal/disclosure/parse.go` | | `go test ./internal/disclosure/ -run TestParse_EntryKind`: fixture splits into preamble/staging/themes **and** round-trips byte-for-byte | Design/Layout | +| T05 | [x] | `Parse` for CONVENTIONS kind (`## Themes` then `## Recent`) | T04 | `internal/disclosure/parse.go` | | `-run TestParse_ConventionKind`: round-trips byte-for-byte; `Staging` == the `## Recent` section | Design/Layout | +| T06 | [x] | `Validate` precondition | T02,T04,T05 | `internal/disclosure/validate.go` | | `-run TestValidate` table test: every matrix row T06 returns its named sentinel; valid + un-migrated return `nil` | Guards §2 | +| T07 | [x] | Invariant: gists ↔ theme files 1:1 | T04 | `internal/disclosure/invariant.go` | | `-run TestInvariant_Pairing`: orphan file → `ErrOrphanThemeFile`; gist w/o file → `ErrMissingThemeFile`; 1:1 → nil; 0↔0 → nil | Invariants | +| T08 | [x] | Invariant: entry in exactly one place | T07 | `internal/disclosure/invariant.go` | | `-run TestInvariant_Uniqueness`: dup across staging+theme → `ErrDuplicateEntry`; dup across two themes → `ErrDuplicateEntry`; single → nil | Invariants | +| T09 | [x] | Invariant: theme links resolve | T08 | `internal/disclosure/invariant.go` | | `-run TestInvariant_Links`: broken link → `ErrBrokenThemeLink`; resolving link → nil | Invariants | +| T10 | [x] | **Layout proof — LEARNINGS** (populated + empty staging) | — | `internal/cli/add/core/insert/layout_learning_test.go` | [P] | `-run TestAdd_LearningLandsAboveThemes`: after `AppendEntry` on a Themes-bearing fixture, `strings.Index(out,newEntry) < strings.Index(out,"## Themes")` — for **both** populated and empty staging | Design/Layout | +| T11 | [x] | **Layout proof — DECISIONS** (populated + empty staging) | — | `internal/cli/add/core/insert/layout_decision_test.go` | [P] | `-run TestAdd_DecisionLandsAboveThemes`: same assertion, both cases | Design/Layout | +| T12 | [x] | **Layout proof — CONVENTIONS** (`## Recent`) | — | `internal/cli/add/core/insert/layout_convention_test.go` | [P] | `-run TestAdd_ConventionLandsInRecent`: after `AppendEntry`, new section index `>` `## Recent` index and `>` `## Themes` index | Design/Layout | +| T13 | [x] | `doc.go` for `internal/disclosure` | T04–T09 | `internal/disclosure/doc.go` | | `make audit` green (doc.go quality floor: behavior-grounded, ~25–100 body lines, related-packages section) | CONVENTIONS/Documentation | +| T14 | [x] | Compliance wiring — invariants vs the real tree | T06–T09 | `internal/compliance/disclosure_test.go` | | `go test ./internal/compliance/ -run TestDisclosureInvariants` green on today's tree (vacuous pass); proven-both-ways via a planted-violation temp fixture that fails it | Invariants | +| T15 | [x] | Milestone gate | T01–T14 | — | | `make lint` = 0 issues; `go test ./...` green; `git diff --stat` shows **no** change to `.context/{LEARNINGS,DECISIONS,CONVENTIONS}.md` | Scope & DoD | + +**Execution waves** (topological order): `T01,T02,T03` ∥ → `T04` → `T05` +→ `T06` → `T07` → `T08` → `T09` → `T13,T14` → `T15`. +`T10,T11,T12` are `[P]` from T01 onward (distinct test files, no shared +edges/sequences) and may run alongside any wave after T01. + +## Epic anchors (TASKS.md projection) + +TASKS.md carries **epic-level anchors only**; this plan is the single +source of truth for milestone progress. The epics **partition** the task +ids — every id in exactly one epic: + +| Epic | Range | Count | +|---|---|---| +| E1 Vocabulary, types, errors | T01–T03 | 3 | +| E2 Root parser + validate | T04–T06 | 3 | +| E3 Cross-file invariants | T07–T09 | 3 | +| E4 Layout proofs (add-path de-risking) | T10–T12 | 3 | +| E5 doc.go, compliance wiring, milestone gate | T13–T15 | 3 | + +Arithmetic: 3 + 3 + 3 + 3 + 3 = **15** = the task count. No id is +double-counted; no id is unclaimed. + +**Completion rule:** an epic is checked `[x]` in TASKS.md only when +every task in its range is `[x]` or `[o]` here. + +## Risks & measurement gates + +- **⚠️ Measurement gate — T10–T12 are the milestone's load-bearing + result.** The entire design rests on "`add` needs zero change because + its anchor always lands above `## Themes`." If any layout proof fails, + the premise is wrong and the spec's Layout section must be revisited + **through `/ctx-plan`** — not patched here. Everything downstream + (M2–M5) reshapes if this fires. +- **CONVENTIONS is the least-verified path.** `AppendAtEnd` was read but + its exact EOF behavior against a trailing `## Recent` section (and any + trailing newline handling) is unproven — T12 is where that lands. +- **`Staging` kept raw** to avoid silent normalization; if M2 later needs + structured staging, that is an amendment, not a redesign. + +## Out of scope (deferred, with pointers) + +| Deferred | Milestone | Note | +|---|---|---| +| The pass itself; gist authoring | M2 | *Gist format ("just enough") TBD graduates to **blocking** here* | +| Real rollout: LEARNINGS → DECISIONS | M3 | Requires M1 guards green | +| CONVENTIONS prose rollout; edits-behind-a-link UX | M4 | *CONVENTIONS staging-parse TBD graduates to blocking here* | +| Suggest-only trigger wiring | M5 | *Growth-nudge threshold TBD graduates to blocking here* | +| Nesting / taxonomy | — | Deferred indefinitely; structure is self-similar (spec Non-Goals) | +| `ctx drift` wiring beyond the existing path check | — | Spec does not require it | + +## Amendments + +| date | what | why | +|---|---|---| +| 2026-07-17 | E4 measurement gate **fired** on T10 (empty-staging case): `ctx learning add` destroyed the `## Themes` section. Root cause was **not** a design flaw but a pre-existing bug — `insert.AfterHeader` truncated the tail (`content[:i]+entry`, dropping `content[i:]`). Fixed on `fix/afterheader-tail-truncation` (spec `specs/fix-afterheader-tail-truncation.md`), merged into this branch; T10–T12 now green with unchanged assertions. Deps T10–T12→`—` (they use only `insert.AppendEntry`, not T01). | The gate did its job: it surfaced a real data-loss defect before any content moved. The design premise ("add needs no accommodation") holds, restated as "add needed one bug fix." No `/ctx-plan` re-open needed — the layout design is sound. | +| 2026-07-17 | Contract refinement: `Parse` is `Parse(content, k) Root` (no error), not `(Root, error)`. Splitting a byte string into segments cannot fail; all refusal lives in `Validate`. An always-nil error return is a smell (and `unparam` would flag it if enabled). Acceptance criteria unaffected (they never referenced the error). Structure decisions are this skill's remit. | Total functions read cleaner and force no dead error-handling on callers. | +| 2026-07-17 | Finding for **M4** (not M1): the real `CONVENTIONS.md` uses `##` headings, not the `###` prose sections the spec's layout assumed. A `##` convention entry collides with the `## Themes`/`## Recent` delimiters. This graduates the "CONVENTIONS staging-parse" TBD to blocking at M4 and likely needs a spec revisit for the conventions layout. M1 unaffected: un-migrated conventions parse as all-preamble and validate vacuously. | Recorded now so M4 starts from reality; the layout proof T12 tested the *design's* `###` assumption, which the real file does not meet. | diff --git a/specs/plans/pd-m2.md b/specs/plans/pd-m2.md new file mode 100644 index 000000000..080a6d9d8 --- /dev/null +++ b/specs/plans/pd-m2.md @@ -0,0 +1,171 @@ +# pd-m2 Plan — the dry-run pass (inspect + propose, move nothing) + +**Spec:** `specs/progressive-disclosure.md` · **Status:** Ready +**Blocking TBDs resolved:** gist format — resolved **in the spec** +(`### Gist format`): one bullet `- <name> — <gist> → [<name>](<noun>/<slug>.md)`, +gist one line ≤~140 chars describing coverage, authored by the pass. + +## Scope & DoD + +Milestone 2 delivers the digesting pass **in dry-run only**: a read-only +CLI that reports a root's staging entries and current themes, and a +skill that reads it, proposes a theme per staging entry (agent-semantic, +human-overridable), authors proposed gists, and shows the plan of what +*would* move — **moving nothing**. The mover (append→verify→remove, gist +write-back) is M3. + +This ordering keeps the risky write path (M3) behind a fully-exercised +read+plan path: the same structured data the mover will consume is built +and tested here first. + +DoD (confirmed by measurement or by the user — never derived from task +completion): + +- [x] `make lint` = 0 issues, `go test ./...` green, `make audit` passes +- [x] `ctx disclosure inspect <file>` reports, for each kind, the staging + entries and current themes; `--json` emits the same structured + (E2) +- [x] Running `ctx disclosure inspect` against any file **writes + nothing** — the file is byte-identical after (E2 acceptance) +- [x] The dry-run skill, exercised on a fixture root, produces a + theme→entries plan with proposed gists and moves/writes nothing + (E3, verified by driving it) +- [x] `git diff` shows **no change** to `.context/{LEARNINGS,DECISIONS, + CONVENTIONS}.md` across the whole milestone + +## Data model & storage + +No persistence. The CLI is a pure read/report over M1's `disclosure.Parse`. +One new value type for structured output: + +```go +// internal/disclosure (types.go) +type Inspection struct { + Kind string // "learning" | "decision" | "convention" + Staging []StagedEntry // un-digested entries, file order + Themes []Theme // current ## Themes (reused from M1) +} +type StagedEntry struct { + Timestamp string + Title string +} +``` + +Kind is inferred from the file's basename (LEARNINGS.md → learning, …) +via a new `disclosure.KindFor(basename) (Kind, bool)`. + +## Contracts + +**Disclosure package additions** (`internal/disclosure`): + +- `KindFor(basename string) (Kind, bool)` — maps a canonical filename to + its Kind; false for anything else. +- `StagedEntries(root Root) []StagedEntry` — the staging zone's entries, + via `heading.ParseEntryBlocks(root.Staging)`. Empty for conventions + (no `## [` entries) — an accepted M2 limitation, see Risks. +- `Inspect(content string, k Kind) Inspection` — Parse + assemble the + Inspection. Total (mirrors Parse). + +**CLI** (`ctx disclosure inspect <file> [--json]`): + +- New command group `ctx disclosure` (hidden=false), one subcommand + `inspect`, registered in `internal/bootstrap/group.go`. `Use` strings + are `config/embed/cmd` constants. +- Reads the file, infers Kind (error if not a canonical knowledge file), + prints a human summary (kind, N staged, list; M themes, list) or + `--json` (the Inspection). **Never writes.** +- Kind-inference failure → a typed error in `internal/err/disclosure`. + +**Skill** (`internal/assets/claude/skills/ctx-digest/SKILL.md`, shipped): + +- Dry-run pass. Steps: run `ctx disclosure inspect --json` on a root → + propose a theme per staged entry (semantic; human can rename/merge/ + override) → author a proposed gist per theme (spec `### Gist format`) + → present the plan (theme → its entries → proposed gist + link). + **Explicitly moves/writes nothing in M2**; ends by naming M3 as the + apply step. Copilot copy synced via `make sync-copilot-skills`. + +No `ctx agent` change (spec Non-Goals). No mover, no write path. + +## Test matrix + +| Behavior / rule | Attempt | Expected | Task | +|---|---|---|---| +| kind inference | "LEARNINGS.md"/"DECISIONS.md"/"CONVENTIONS.md" | KindFor → (kind, true) | T01 | +| kind inference | "README.md" | KindFor → (_, false) | T01 | +| staged entries listed | entry-kind root with N staged | StagedEntries → N in file order | T02 | +| staged entries: empty staging | migrated root, empty staging | StagedEntries → nil | T02 | +| Inspect assembles | fixture root | kind+staging+themes match Parse | T03 | +| inspect reports | `inspect LEARNINGS.md` on a fixture | human output lists staged + themes | T05 | +| inspect --json | `inspect --json` | valid JSON decodes to Inspection | T06 | +| inspect writes nothing | `inspect` on a real root | file byte-identical after | T07 | +| inspect rejects non-knowledge file | `inspect README.md` | non-zero exit, typed error | T08 | +| dry-run moves nothing | drive the skill on a fixture | fixture byte-identical; plan produced | T12 | + +## Task breakdown + +| id | st | task | deps | files | [P] | acceptance criterion | spec ref | +|---|---|---|---|---|---|---|---| +| T01 | [x] | `KindFor` filename→Kind | — | `internal/disclosure/kind.go`, `kind_test.go` | [P] | `go test ./internal/disclosure/ -run TestKindFor`: three canonical names map true; others false | Contracts | +| T02 | [x] | `StagedEntry` type + `StagedEntries` | — | `internal/disclosure/types.go`, `inspect.go`, `inspect_test.go` | | `-run TestStagedEntries`: N entries in order; empty staging → nil | Data model | +| T03 | [x] | `Inspection` type + `Inspect` | T01,T02 | `internal/disclosure/types.go`, `inspect.go` | | `-run TestInspect`: kind/staging/themes match a Parse of the same fixture | Contracts | +| T04 | [x] | `Use` constants + kind-inference error sentinel | — | `internal/config/embed/cmd/*.go`, `internal/err/disclosure/*.go`, `commands/text/errors.yaml` | [P] | `go test ./internal/audit/ -run TestDescKeyYAMLLinkage` green (bijection holds) | CONVENTIONS | +| T05 | [x] | `ctx disclosure inspect` group + human output | T03,T04 | `internal/cli/disclosure/**`, `internal/bootstrap/group.go` | | `ctx disclosure inspect <fixture>` prints kind, staged list, theme list | CLI | +| T06 | [x] | `--json` output | T05 | `internal/cli/disclosure/cmd/inspect/run.go` | | `inspect --json <fixture>` output `json.Unmarshal`s into Inspection with expected values | CLI | +| T07 | [x] | write-nothing guarantee | T05 | `internal/cli/disclosure/cmd/inspect/run_test.go` | | integration test: file bytes before == after `inspect` | Scope/DoD | +| T08 | [x] | reject non-knowledge file | T05 | `internal/cli/disclosure/cmd/inspect/run.go`, `run_test.go` | | `inspect README.md` exits non-zero with the kind-inference sentinel | Contracts | +| T09 | [x] | `doc.go` for the CLI group + disclosure additions | T05 | `internal/cli/disclosure/doc.go`, `cmd/**/doc.go` | | `make audit` doc.go + docstring floors pass | CONVENTIONS | +| T10 | [x] | command wiring guard stays green | T05 | — | | `go test ./internal/compliance/ -run TestShippedHooksResolve` and command-tree tests green with the new group | CLI | +| T11 | [x] |XX `ctx-digest` skill (dry-run) SKILL.md | T06 | `internal/assets/claude/skills/ctx-digest/SKILL.md` | | skill frontmatter valid (`go test ./internal/assets/... -run Frontmatter`); body describes inspect→propose→gist→plan and states "moves nothing (M2); apply is M3" | Skill | +| T12 | [x] |XX **dry-run drive** (measurement) | T11 | scratchpad fixture | | drive the skill on a fixture root: a theme→entries plan with gists is produced AND the fixture is byte-identical after | Scope/DoD | +| T13 | [x] |XX copilot skill sync | T11 | `internal/assets/integrations/copilot-cli/skills/ctx-digest/**` | | `make check-copilot-skills` green | Skill | +| T14 | [x] |XX milestone gate | T01–T13 | — | | `make lint` 0, `go test ./...` green, `make audit` pass; canonical `.md` byte-identical | Scope/DoD | + +**Execution waves:** `T01,T02,T04` ∥ → `T03` → `T05` → `T06,T07,T08` → +`T09,T10` → `T11` → `T12,T13` → `T14`. + +## Epic anchors (TASKS.md projection) + +Epics **partition** the task ids: + +| Epic | Range | Count | +|---|---|---| +| E1 Disclosure inspect model | T01–T04 | 4 | +| E2 `ctx disclosure inspect` CLI | T05–T10 | 6 | +| E3 Dry-run skill | T11–T14 | 4 | + +Arithmetic: 4 + 6 + 4 = **14** = the task count. No id double-counted or +unclaimed. **Completion rule:** an epic is `[x]` in TASKS.md only when +every task in its range is `[x]`/`[o]` here. + +## Risks & measurement gates + +- **⚠️ T12 is the milestone's measurement gate.** If the dry-run cannot + produce a coherent theme→entries plan from the inspect output, the + skill's design is wrong and should be reconsidered before M3 builds the + mover on top of it. +- **CONVENTIONS staging is empty under the entry-based model.** Real + CONVENTIONS uses `##` sections, not `## [` entries, so `StagedEntries` + returns nil for it — inspect reports zero staged conventions. This is + the same `##`-vs-`###` gap flagged in pd-m1's amendments; conventions + digestion is genuinely an **M4** problem. M2 inspect still works for + conventions (kind + themes), it just lists no staged entries. +- **New user-facing CLI surface.** `ctx disclosure` is a new command; + the out-of-band `_ctx-surface-audit` will want docs/cli coverage — + tracked for the release pass, not M2 (spec adds no docs requirement). + +## Out of scope (deferred, with pointers) + +| Deferred | Milestone | Note | +|---|---|---| +| The mover: append→verify→remove, gist write-back | M3 | consumes the same Inspection this milestone builds | +| First real rollout LEARNINGS→DECISIONS | M3 | requires the mover | +| CONVENTIONS digestion (`##` model) | M4 | *`##`-vs-`###` TBD is blocking here* | +| Theme taxonomy / naming discipline | M3 | agent proposes at runtime in M2 dry-run; matters when files are created (M3) | +| Suggest-only trigger wiring | M5 | *nudge-threshold TBD blocking here* | + +## Amendments + +| date | what | why | +|---|---|---| +| — | — | — | diff --git a/specs/progressive-disclosure.md b/specs/progressive-disclosure.md new file mode 100644 index 000000000..3087ec167 --- /dev/null +++ b/specs/progressive-disclosure.md @@ -0,0 +1,204 @@ +# Spec: progressive disclosure for canonical knowledge files + +> Design brief: `/ctx-brainstorm`, session 87e465a0, 2026-07-16. +> Decision: DECISIONS.md `[2026-07-16-215955]`. +> Builds on — does not re-litigate — `specs/computed-index-projection.md` +> (`ctx index`, the cheap heading projection, and its Non-Goals). + +## Problem + +Canonical knowledge files grow without bound, and the entries stay +**valid** — so nothing can be dropped. Time-sharding plus a +load-excluded "cold" bucket is already rejected (a supersession pass +found ~1.5% cold across 162 entries; recency-gating is dangerous because +old ≈ live). + +At scale this breaks a real workflow: an agent that legitimately wants +system understanding reads every decision, then every learning, and +exhausts its context window. Two existing levers are insufficient: + +- **Consumption discipline** (headings-first via `ctx index`) is + *necessary but not sufficient*: an agent can always choose to read the + whole file, and will when it wants completeness. +- **Consolidation** does not help: the 2026-07-16 pass moved LEARNINGS + only 98 → 88, because the corpus is dense with *distinct signal*, not + redundancy. + +The missing piece is **lossless tiering**: compress history into a +compact top layer, keep every body reachable, and descend only as the +task demands. + +## Design + +### Three self-similar tiers + +- **Tier 0 — the root** (`LEARNINGS.md`, `DECISIONS.md`, + `CONVENTIONS.md`): *bounded*. Preamble + a **staging zone** + a + `## Themes` section carrying, per theme, a "just enough" gist and a + markdown link to its theme file. +- **Tier 1 — theme files** (`.context/learnings/<theme>.md`, + `.context/decisions/<theme>.md`, `.context/conventions/<theme>.md`): + the entry bodies for that theme. Reachable **only** via the root's + links — every artifact is reachable from the canonical file by + following links, however many hops. +- **Tier 2+ — recursion (deferred)**: an overgrown theme file becomes a + root in its own right (sub-theme gists + its own staging), handled by + the same pass. Taxonomy emerges only when the corpus demands it; + nesting is **not precluded**, just not built. + +Reading the root **alone** yields compressed history **+** verbatim +recent delta = a complete current picture, with **no staleness gap**, +because staging *is* the un-digested remainder by construction. + +### Gist format (resolves the M2-blocking "just enough" TBD) + +Each theme is exactly one bullet line under `## Themes`: + +``` +- <theme-name> — <one-line gist> → [<theme-name>](<noun>/<slug>.md) +``` + +- `<theme-name>`: a short kebab-or-words label (e.g. `hooks`, `error + handling`). +- `<one-line gist>`: **one line, a soft ceiling of ~140 chars**, saying + what the theme *covers* — the shape of its knowledge, not a list of + its entries ("hook mechanics: output channels, key names, compliance + wiring", not "entry A; entry B; entry C"). It conveys *whether to + drill*, nothing more. +- The separator is the em-dash metadata separator (`token.MetaSeparator`) + before the gist and ` → ` before the link; the link target is + `<noun>/<slug>.md` relative to the context dir. This is exactly the + shape `disclosure.parseThemeBullet` already parses. + +The gist is **authored by the pass** (an LLM summarizing the theme's +entries), regenerated whenever the theme gains entries. It is stored +(not recomputed on read) precisely because it is expensive to produce — +the reconciling rationale in DECISIONS `[2026-07-16-215955]`. + +### Layout (forced by the existing write path) + +`ctx X add` must not change. Verified anchors: + +| File | Insert | Staging must sit | +|---|---|---| +| DECISIONS, LEARNINGS | `beforeFirstEntry`: before the first line-start `## [`; falls back to `AfterHeader` when none | **above** `## Themes` | +| CONVENTIONS | `AppendAtEnd` (EOF) | **below**, in a trailing `## Recent` | + +``` +LEARNINGS.md / DECISIONS.md CONVENTIONS.md +# Learnings # Conventions +<!-- UPDATE WHEN … --> <!-- … --> + ## Themes +## [ts] newest ← add (STAGING) - naming — gist… → link +## [ts] entry - output — gist… → link +## Themes ## Recent + - hooks — gist… → link ### new convention ← add (STAGING) + - output — gist… → link +``` + +Because the fallback is `AfterHeader`, an entry lands above `## Themes` +**even when staging is empty**. CONVENTIONS needs the explicit +`## Recent` heading because `###` prose sections would otherwise nest +ambiguously *inside* `## Themes`. + +### Consequence: no consumption rewire + +Because the root itself is bounded, the existing `ctx agent` packet +("Read These Files: …") becomes safe automatically. No packet change. +The only doc change: the playbook notes "drill into theme files as the +task demands." + +### The pass (a new skill — the deliverable) + +Agent-driven, human-gated, never inline in another ceremony: + +1. Read the staging zone. +2. Propose a theme per staged entry — the agent suggests semantically; + the human may override, rename, or supply themes. +3. Per target theme: **append body to the theme file → verify + byte-presence → only then remove from staging**. +4. Regenerate the gist of **every theme it touched**; leave untouched + themes alone. +5. Create `## Themes` (and `## Recent` for conventions) on first run. + +### Triggers — suggestion only + +The growth/threshold nudge, `/ctx-remember`, and `/ctx-wrap-up` may +**suggest** the pass. None of them perform it. Wrap-up especially must +stay light: the human is closing the laptop to go live their life, and +semantic work there is against their interest. + +## Guards + +1. **Append → verify → remove.** Never remove-then-append. Any verify + failure aborts the whole pass with the root untouched. +2. **Precondition validate** (`index.Validate`-style): **zero or one** + `## Themes`; no `## [` below it; staging parses into discrete + entries. Refuse and fail loud otherwise. **Never regenerate from + "what I recognized"** — that was the exact root cause of the original + clobber bug (unparsed content treated as empty). + + Zero `## Themes` means the root is **not yet migrated**: this is the + first run, and the pass creates the section (see step 5 of the pass). + Two or more is malformed → refuse. Accepting zero is what keeps + un-migrated roots passing from day one, so the gate is a signal rather + than noise that trains people to ignore it. The *invariants* need no + such carve-out: "no `## [` below `## Themes`" and "gists ↔ theme files + 1:1" are vacuously true on an un-migrated root. +3. **Crash ordering**: theme-file appends (additive) first, then one + root rewrite. Worst case = duplication (detectable, recoverable), + never loss. +4. **Fail loud, no auto-repair** — matching the learning-add clobber-fix + precedent. + +## Invariants (mechanically checkable) + +- No line-start `## [` below `## Themes` in a root. +- Root gists ↔ theme files are 1:1 (no orphan file; no gist without a + file). +- Every theme link resolves (existing `ctx drift` path check). +- Every entry lives in **exactly one** place: staging XOR one theme file. + +## Tests + +- **Invariant compliance tests** for each rule above. +- **Conservation**: `staging_before == moved + staging_after`; every + moved body byte-present in exactly one theme file; zero loss, zero + dups. +- **`add` still works**: `ctx learning add` with populated *and* empty + staging both land above `## Themes`; `ctx convention add` lands inside + `## Recent`. +- **Abort**: corrupt the root → pass refuses, file byte-identical. +- **Idempotency**: pass with empty staging = no-op. + +## Acceptance + +- Each in-scope root stays bounded: gists + links + staging only. +- An agent reading only a root can describe what the corpus says and + knows where to drill. +- No entry is lost or duplicated across a pass; guards refuse on + malformed input. +- `ctx decision/learning/convention add` work unchanged, with zero code + edits to the add path. +- The pass is codified as a reusable skill. + +## Non-Goals + +- **No time-sharding, no recency-gating, no load-excluded cold bucket** — + settled in `specs/computed-index-projection.md`. +- **No change to the `add` write path.** +- **No `ctx agent` packet rewire** — boundedness makes it unnecessary. +- **No taxonomy/nesting machinery now** — the structure is self-similar, + so nesting is available when themes outgrow their file. +- **CONSTITUTION.md and TASKS.md are out of scope** — the former is small + by design, the latter is auto-archived. +- **KB pipeline untouched.** + +## Phasing (sketch — refine via /ctx-task-out) + +1. Guards + invariants + the structural vocabulary (`## Themes`, + `## Recent`), with tests, before any content moves. +2. The pass as a skill, dry-run first (propose themes, move nothing). +3. First real rollout on LEARNINGS (largest corpus), then DECISIONS. +4. CONVENTIONS (prose model, `## Recent`, edits-behind-a-link UX). +5. Wire the suggest-only triggers.