Redesign RTK integration to be net-positive on tokens and quality

Shift from "always prefix every command with rtk, use ultra-compact for
maximum savings" to a selective, lossless-by-default policy so RTK cuts
tokens without hiding detail the model needs — detail loss otherwise forces
re-runs that cost more than they save and degrade output quality.

- AGENTS.md: lean, always-on policy. Compress noisy/low-stakes output; keep
  full fidelity for diffs/patches, structured/JSON output, secrets, small
  output, and files to edit; never default to lossy modes; escalate fidelity
  (not blind re-runs) when a compressed view falls short.
- SKILL.md: recommend RTK's auto-rewrite PreToolUse hook (rtk init -g) over
  manual prefixing; add a fidelity ladder, when-NOT-to-use, and net-savings
  guidance; bump to 0.2.0.
- references/commands.md: reorganize into green/yellow/red tiers (compress /
  plain mode only / run raw); fix rtk-read vs native Read; drop
  aggressive-as-default. Leaner, so it costs less always-on context.
- references/analytics.md: optimize NET savings — watch `rtk gain --failures`
  and `rtk discover`; stop wrapping poor fits.
- README.md: correct the "no auto-rewrite hook" framing, add a "when RTK helps
  vs keep full fidelity" rationale grounded in context-rot research, reframe
  the teaching table by tier.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VvpBXNCbbg4eibmZQZAbb6

Author: claude

AGENTS.md

@@ -1,32 +1,48 @@
 # RTK Token Optimization for Command Code
 
-RTK (Rust Token Killer) is installed on this system. It reduces LLM token
-consumption by 60-90% by filtering and compressing shell command output.
+RTK (Rust Token Killer) compresses **shell command output** before it enters the
+context window. Used well it cuts tokens on noisy commands *and* sharpens context
+(less noise → better reasoning). Used badly — compressing output you actually
+needed — it hides detail and forces re-runs that cost *more* than they save.
+These rules keep it net-positive.
 
-## Prerequisite
+## Setup
 
-RTK must be installed and in PATH. Verify with: `rtk --version`
+RTK is optional. If `rtk --version` fails, run commands normally — never block
+work to install it. Best setup: install RTK's auto-rewrite hook once with
+`rtk init -g` (then restart Command Code) so Bash commands are rewritten to `rtk`
+automatically and you never prefix by hand. Without a hook, prefix manually per
+the tiers in @references/commands.md.
 
-If not installed, direct the user to https://github.com/rtk-ai/rtk#installation.
+## The one rule: compress noise, preserve signal
 
-## Core Rule
+Use `rtk <cmd>` when output is **large, repetitive, low-stakes** — skim, not
+study: `rtk ls`, `rtk tree`, `rtk git status`, `rtk git log`, `rtk docker ps`,
+`rtk pip list`, and big test/build runs (`rtk cargo test`, or `rtk err <cmd>` —
+RTK keeps the failures and drops the green).
 
-Always prefix supported shell commands with `rtk`. Instead of running
-`git status`, run `rtk git status`. Instead of `ls -la`, run `rtk ls`.
+Run the command **raw (no `rtk`)** when you need every character:
 
-@references/commands.md
-@references/analytics.md
+- a **diff/patch you'll apply**, or a hunk you'll edit — exact bytes and line numbers matter
+- **structured output you'll parse** — JSON, `--format=…`, anything piped into another tool
+- **small output** (≲30 lines) — nothing to save, and real risk of dropping the one line you need
+- **secrets / exact config** — never reason about a lossy view of these
+- a file you intend to **edit** — use the native Read tool (lossless; it bypasses RTK anyway)
+
+When unsure, start raw. Lean context comes from cutting *noise*, not *signal*.
 
-## Ultra-Compact Mode
+## Never default to lossy modes
 
-Add `-u` or `--ultra-compact` for maximum savings:
-```bash
-rtk git status -u
-```
+Plain `rtk <cmd>` is near-lossless (keeps errors, diffs, stack traces, exit
+codes). `-u` / `--ultra-compact`, `rtk read … -l aggressive`, and `rtk smart`
+(2-line summary) are **lossy** — opt-in only for skimming something huge and
+unimportant, never your default.
 
-## Exclusions
+## If a compressed view isn't enough
 
-- Agent built-in file tools (non-Bash) do not pass through RTK
-- Use shell equivalents: `rtk read <file>`, `rtk grep <pattern>`, `rtk find <pattern>`
-- Piped commands (`|`) and heredocs (`<<`) bypass RTK rewrite
-- Commands already prefixed with `rtk` are left as-is
+RTK retains the full unfiltered output, so you don't have to guess and you
+shouldn't thrash: re-run that one command raw (or `rtk proxy <cmd>`) to see
+everything, then move on. One deliberate re-run is fine; blind repeated re-runs
+erase the savings.
+
+@references/commands.md

README.md

@@ -9,8 +9,11 @@ learns your coding style.
 ## How It Works
 
 Command Code CLI reads `~/.commandcode/AGENTS.md` and injects it into the system
-prompt for every session. This repository provides that AGENTS.md file, teaching
-Command Code to prefix shell commands with `rtk` for automatic output compression.
+prompt for every session. This repository provides that AGENTS.md file (plus a
+loadable skill), teaching Command Code to route **noisy** shell output through
+`rtk` while keeping **full fidelity** for output it needs exactly. The fastest
+setup is RTK's auto-rewrite hook (`rtk init -g`), which rewrites commands for you
+so nothing has to be prefixed by hand.
 
 ```
 Without RTK:                                 With RTK:
@@ -20,6 +23,37 @@ cargo test (25,000 tokens)                   rtk cargo test (2,500 tokens)
 ls -la (800 tokens)                          rtk ls (150 tokens)
 ```
 
+## When RTK helps — and when to keep full fidelity
+
+RTK only pays off when it removes **noise**. This integration is deliberately
+selective:
+
+**Compress (🟢/🟡)** — large, repetitive, low-stakes output you skim: listings,
+`git status` / `log`, dependency installs, container/cluster status, and big
+test/build runs (plain `rtk` keeps the failures and drops the green).
+
+**Keep full fidelity (🔴)** — run the bare command for diffs/patches you'll apply,
+JSON or `--format` output you'll parse, secrets, small outputs, and files you'll
+edit (use the agent's native file tools). Compressing these is exactly what makes
+a tool like this *cost* tokens — the model loses the detail and re-runs the
+command.
+
+Two design choices keep it net-positive:
+
+- **Lossless by default.** Plain `rtk <cmd>` preserves errors, stack traces, diff
+  hunks, and exit codes; the lossy modes (`-u` / `--ultra-compact`, `-l
+  aggressive`, `rtk smart`) are opt-in for skimming only — never the default. On
+  output it can't parse, RTK falls back to the full raw text.
+- **Measure net, not gross.** `rtk gain` reports gross savings; the goal is *net*
+  — savings minus any re-runs and minus the standing cost of these instructions.
+  `rtk gain --failures` and `rtk discover` show where RTK fits and where it
+  doesn't (see [references/analytics.md](references/analytics.md)).
+
+This matters for quality too: every frontier model degrades as irrelevant context
+grows ("context rot" / "lost in the middle"), so cutting genuine noise can *help*
+reasoning — while over-compressing real signal hurts it. The rules above aim for
+the first and avoid the second.
+
 ## Installation
 
 ### Prerequisites
@@ -36,6 +70,11 @@ ls -la (800 tokens)                          rtk ls (150 tokens)
 > enforcement, install the memory (Method 2); the skill (Method 1) is the
 > quickest install and is enough when you mainly want it during shell-heavy work.
 
+> **Even better — the auto-rewrite hook.** RTK can install a `PreToolUse` hook
+> (`rtk init -g`) that rewrites Bash commands to `rtk` automatically, so the agent
+> never prefixes anything by hand. Pair it with the memory or skill below, which
+> carry the *when-to-compress* rules. See `rtk init --help`.
+
 ### Method 1: Skill (quick install, on-demand)
 
 ```bash
@@ -81,22 +120,24 @@ cp references/analytics.md ~/.commandcode/references/analytics.md
 
 ## What It Teaches the Agent
 
-Once installed, Command Code will automatically use RTK-optimized commands:
+Once installed, Command Code routes **noisy, low-stakes** commands through RTK and
+leaves precise output alone:
 
-| Category | Without RTK | With RTK | Savings |
+| Category | Command | Through RTK? | Est. savings |
 |---|---|---|---|
-| Git | `git status` | `rtk git status` | ~80% |
-| Files | `cat file.rs` | `rtk read file.rs` | ~70% |
-| Search | `grep -r "pattern" .` | `rtk grep "pattern" .` | ~80% |
-| Tests | `cargo test` | `rtk cargo test` | ~90% |
-| Build | `cargo build` | `rtk cargo build` | ~80% |
-| Docker | `docker ps` | `rtk docker ps` | ~80% |
-| GitHub | `gh pr list` | `rtk gh pr list` | ~80% |
+| Status / listings | `rtk git status`, `rtk ls`, `rtk tree` | 🟢 yes | ~80% |
+| Logs / containers | `rtk docker ps`, `rtk log app.log` | 🟢 yes | ~80% |
+| Dependencies | `rtk pip list`, `rtk pnpm list` | 🟢 yes | ~70% |
+| Tests / build | `rtk cargo test`, `rtk err <cmd>` | 🟡 plain mode (keeps failures) | ~90% |
+| Diffs you'll apply | `git diff`, `git show` | 🔴 run raw | — |
+| JSON / parsed output | `… --format json` | 🔴 run raw | — |
+| Files you'll edit | native Read tool | 🔴 not RTK | — |
 
-_Savings are illustrative estimates; actual numbers vary by command and output
-size. Run `rtk gain` to measure your own._
+_Savings are illustrative; actual numbers vary by command and output size. Run
+`rtk gain` to measure your own — and `rtk gain --failures` to spot poor fits._
 
-See [references/commands.md](references/commands.md) for the full list.
+See [references/commands.md](references/commands.md) for the full tiered list and
+[references/analytics.md](references/analytics.md) for measuring net savings.
 
 ## Verify
 
@@ -106,7 +147,10 @@ rtk gain --graph        # Visual savings chart
 ```
 
 After running a few commands through Command Code, `rtk gain` will show the
-accumulated savings.
+accumulated savings. Track **net** savings, not just the headline number:
+`rtk gain --failures` lists commands RTK had to pass through raw (poor fits), and
+`rtk discover` finds new high-value targets. See
+[references/analytics.md](references/analytics.md).
 
 ## Files
 

SKILL.md

@@ -1,17 +1,19 @@
 ---
 name: rtk-token-optimizer
 description: >-
-  Compress shell command output to cut LLM token usage 60-90% by prefixing
-  commands with `rtk`. Use whenever running shell commands in Command Code —
-  git, cargo, npm/pnpm, pytest/jest, docker, kubectl, gh, grep, find, ls, cat,
-  build and lint tools — so large outputs are filtered before entering context.
+  Cut LLM token usage on noisy shell output by routing large, low-stakes
+  commands (listings, status, logs, dependency installs, test/build runs)
+  through RTK — while keeping full fidelity for diffs, structured/JSON output,
+  errors you're debugging, and anything you'll parse or edit. Use when running
+  shell commands in Command Code that produce big, repetitive output.
 license: Apache-2.0
 compatibility: >-
-  Requires the RTK binary (>= 0.42.0) installed and on PATH. Verify with
-  `rtk --version`. Install from https://github.com/rtk-ai/rtk
+  Requires the RTK binary (>= 0.42.0) on PATH. Verify with `rtk --version`.
+  Install from https://github.com/rtk-ai/rtk. RTK is optional — run commands
+  normally if it isn't installed.
 metadata:
   author: Coding-Dev-Tools
-  version: "0.1.0"
+  version: "0.2.0"
   homepage: https://github.com/rtk-ai/rtk
 allowed-tools: Bash(rtk:*) Bash(git:*) Bash(cargo:*) Bash(ls:*) Bash(cat:*) Bash(grep:*) Bash(find:*) Bash(diff:*) Bash(docker:*) Bash(kubectl:*) Bash(gh:*) Bash(glab:*) Bash(pnpm:*) Bash(npm:*) Bash(pip:*) Bash(bundle:*) Bash(ruff:*) Bash(tsc:*) Bash(eslint:*) Bash(pytest:*) Bash(go:*) Bash(jest:*) Bash(vitest:*) Bash(dotnet:*) Bash(aws:*) Bash(psql:*) Bash(prisma:*) Bash(wget:*)
 ---
@@ -20,67 +22,91 @@ allowed-tools: Bash(rtk:*) Bash(git:*) Bash(cargo:*) Bash(ls:*) Bash(cat:*) Bash
 
 ## What it is
 
-[RTK](https://github.com/rtk-ai/rtk) (Rust Token Killer) is a CLI proxy that
-filters and compresses shell command output before it reaches the context
-window — removing noise, grouping similar items, truncating redundancy, and
-deduplicating repeated lines. Typical savings are 60-90% on common dev
-commands. RTK ships as a single self-contained Rust binary that adds minimal
-per-command overhead.
+[RTK](https://github.com/rtk-ai/rtk) (Rust Token Killer) is a single Rust binary
+that filters **shell command output** before it reaches the context window —
+dropping noise (progress bars, passing tests, decorative formatting, repeated log
+lines) while keeping signal (errors, stack traces, diff hunks, exit codes). When
+it can't parse a command's output it falls back to the full raw text, so it never
+silently eats data. Typical savings are 60–90% on noisy commands, at <10 ms
+overhead.
 
-## How it works
+Done right this is a **double win**: fewer tokens *and* a leaner context, which
+measurably improves model reasoning — every frontier model degrades as irrelevant
+context grows ("context rot" / "lost in the middle"). Done wrong —
+over-compressing output you actually needed — it hides detail and triggers
+re-runs that cost more than they saved. The rest of this skill is how to stay on
+the winning side.
 
-```
-You run:        git status
-RTK rewrites:   rtk git status
-RTK filters:    smart filtering · grouping · truncation · deduplication
-Result:         ~200 tokens enter context instead of ~2,000
+## Install — two ways
+
+### 1. Auto-rewrite hook (recommended)
+
+Let a `PreToolUse` hook rewrite Bash commands to their `rtk` equivalents
+automatically, so you never prefix by hand and nothing gets forgotten. RTK ships
+this for natively-supported agents:
+
+```bash
+rtk init -g          # installs the PreToolUse rewrite hook, then restart the agent
 ```
 
-## Core rule
+RTK natively targets Claude Code, Copilot, Cursor, Gemini, Cline, and more.
+Command Code also supports `PreToolUse` hooks — register one that pipes the Bash
+command through RTK's rewrite (see the Command Code hooks docs and `rtk init
+--help`). With a hook installed you **run normal commands**, and only need the
+fidelity rules below for the cases where you want to *bypass* compression.
 
-Prefix supported shell commands with `rtk`. Run `rtk git status` instead of
-`git status`, `rtk cargo test` instead of `cargo test`, `rtk read file.rs`
-instead of `cat file.rs`.
+### 2. Manual prefixing (no hook)
 
-Most-used commands:
+Without a hook, prefix commands yourself per the tiers in
+[references/commands.md](references/commands.md). Reliable, but you have to
+remember it — and `|` pipes and `<<` heredocs bypass the rewrite.
 
-| Instead of | Use |
-|---|---|
-| `git status` / `git diff` | `rtk git status` / `rtk git diff` |
-| `ls -la` | `rtk ls` |
-| `cat file` | `rtk read file` |
-| `grep -r "x" .` | `rtk grep "x" .` |
-| `cargo test` / `pytest` / `npm test` | `rtk cargo test` / `rtk pytest` / `rtk jest` |
-| `docker ps` | `rtk docker ps` |
+## Decision rule: compress noise, preserve signal
 
-Full command list: [references/commands.md](references/commands.md).
-Savings analytics: [references/analytics.md](references/analytics.md).
+- 🟢 **Compress freely** — large, noisy, low-stakes output you skim:
+  `rtk ls`, `rtk git status`, `rtk git log`, `rtk docker ps`, `rtk pip list`.
+- 🟡 **Default mode only** — big runs where you need the failures: `rtk cargo
+  test`, `rtk err <cmd>`. Plain `rtk` keeps errors/diffs — don't add `-u`.
+- 🔴 **Keep full fidelity (run raw)** — diffs/patches you'll apply, JSON or
+  `--format` output you'll parse, secrets, small outputs, and files you'll edit
+  (use the native Read tool).
 
-## Prerequisite
+Full tiered table: [references/commands.md](references/commands.md).
 
-RTK must be on PATH. If `rtk --version` fails, install it:
+## Fidelity ladder
 
-- **macOS:** `brew install rtk`
-- **Linux/macOS:** `curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/refs/heads/master/install.sh | sh`
-- **Windows:** download `rtk-x86_64-pc-windows-msvc.zip` from the [releases page](https://github.com/rtk-ai/rtk/releases) and put `rtk.exe` on PATH.
+Use the *least* compression that still answers the question:
 
-## Ultra-compact mode
+```
+raw / native Read  →  rtk <cmd> (near-lossless, default)  →  -u / -l aggressive / rtk smart (lossy, skim-only)
+```
 
-Add `-u` / `--ultra-compact` for maximum savings: `rtk git status -u`.
+Start as far left as the task needs. Escalate compression only for big, boring
+output; escalate *fidelity* (drop back to raw or `rtk proxy <cmd>`) the moment a
+compressed view is missing something — once, deliberately, not by re-running
+blindly.
 
 ## When NOT to use RTK
 
-- RTK filters **command output**, not conversation messages.
-- Piped commands (`|`) and heredocs (`<<`) bypass the rewrite.
-- No auto-rewrite hook in Command Code — prefix each command with `rtk` explicitly.
+- Diffs/patches you'll apply, JSON/`--format` you'll parse, secrets, small
+  outputs — run raw.
+- Files you'll edit — use the native Read tool (lossless; bypasses RTK anyway).
+- It filters command output, not conversation messages; `|` and `<<` bypass the
+  hook.
 
-## Verify savings
+## Measure net savings
 
-```bash
-rtk gain            # session summary
-rtk gain --graph    # 30-day savings chart
-```
+`rtk gain` shows gross savings; `rtk gain --failures` shows what passed through
+raw; `rtk discover` finds good new targets. Optimize **net** tokens (savings
+minus re-runs), not the headline number. Full reference:
+[references/analytics.md](references/analytics.md).
+
+## Prerequisite
+
+RTK on PATH (`rtk --version`). If missing:
+
+- **macOS:** `brew install rtk`
+- **Linux/macOS:** `curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/refs/heads/master/install.sh | sh`
+- **Windows:** download `rtk-x86_64-pc-windows-msvc.zip` from the [releases page](https://github.com/rtk-ai/rtk/releases) and put `rtk.exe` on PATH.
 
-Savings vary by command and output size; `rtk gain` reports your actual
-numbers. See [references/analytics.md](references/analytics.md) for the full
-analytics reference.
+RTK is optional; never block work to install it.

references/analytics.md

@@ -1,24 +1,30 @@
-# RTK Analytics
+# RTK Analytics — measure *net* savings, not just gross
 
-Quantify token savings with `rtk gain`, and find more opportunities with
-`rtk discover`. This is the canonical analytics reference (command rewrites
-live in [commands.md](commands.md)).
+`rtk gain` reports **gross** tokens saved. The number that actually matters is
+**net**: gross savings minus (a) tokens spent re-running commands when a
+compressed view hid something, and (b) the standing cost of these instructions in
+context. Optimize for net.
 
-| Command | Description |
+| Command | Use it to |
 |---|---|
-| `rtk gain` | Session summary: tokens saved, efficiency meter |
-| `rtk gain --graph` | ASCII bar chart of daily savings (last 30 days) |
-| `rtk gain --history` | Recent command history with per-command savings |
-| `rtk gain --daily` | Day-by-day breakdown |
-| `rtk gain --weekly` | Weekly breakdown |
-| `rtk gain --monthly` | Monthly breakdown |
-| `rtk gain --all --format json` | JSON export for dashboards |
-| `rtk gain --quota` | Monthly quota savings estimate |
-| `rtk gain --failures` | Commands that bypassed RTK (parse-failure log) |
-| `rtk gain --reset --yes` | Reset all saved-token counters to zero |
-| `rtk discover` | Find commands that could benefit from RTK |
-| `rtk discover --all --since 7` | All projects, last 7 days |
+| `rtk gain` | session summary: tokens saved, efficiency |
+| `rtk gain --graph` | 30-day savings trend |
+| `rtk gain --history` | per-command savings — see where RTK actually pays off |
+| `rtk gain --failures` | commands RTK **couldn't parse** and passed through raw |
+| `rtk discover` | find *good* new opportunities (don't blanket-apply) |
 | `rtk session` | RTK adoption across recent sessions |
+| `rtk gain --all --format json` | export for dashboards (run raw if you'll parse it) |
 
-Run `rtk gain` periodically to quantify actual savings; numbers vary by
-command and output size.
+## Reading the signal
+
+- **High `--history` savings on noisy commands** → working as intended; keep going.
+- **Entries in `--failures`** → RTK fell back to raw output for those, so they
+  aren't saving — and may be a poor fit (structured or edge-case commands). Run
+  them raw and stop wrapping them.
+- **You re-ran a command raw right after its `rtk` version** → that pair was a net
+  *loss*. Note the command type and stop compressing it.
+- **`rtk discover`** surfaces high-volume, noisy commands worth wrapping — a far
+  better guide than wrapping everything by reflex.
+
+Savings vary by command and output size; let `rtk gain` show your real numbers
+rather than assuming the headline 60–90%.