From d0247f2e2f04eccf595556b7076cda5a5c6a2b5a Mon Sep 17 00:00:00 2001 From: luoxianzi Date: Tue, 7 Jul 2026 11:10:28 +0800 Subject: [PATCH] =?UTF-8?q?Add=20praetor=20=E2=80=94=20Claude=20plans,=20C?= =?UTF-8?q?odex=20executes,=20a=20binding=20judge=20verifies=20(Workflow?= =?UTF-8?q?=20Orchestration)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 1 + plugins/praetor/.claude-plugin/plugin.json | 20 +++ plugins/praetor/LICENSE | 21 ++++ plugins/praetor/README.md | 115 ++++++++++++++++++ plugins/praetor/agents/codex-judge.md | 29 +++++ plugins/praetor/commands/delegate.md | 10 ++ .../skills/dispatching-to-codex/SKILL.md | 49 ++++++++ .../skills/dispatching-to-codex/preflight.sh | 46 +++++++ .../skills/writing-codex-briefs/SKILL.md | 50 ++++++++ 9 files changed, 341 insertions(+) create mode 100644 plugins/praetor/.claude-plugin/plugin.json create mode 100644 plugins/praetor/LICENSE create mode 100644 plugins/praetor/README.md create mode 100644 plugins/praetor/agents/codex-judge.md create mode 100644 plugins/praetor/commands/delegate.md create mode 100644 plugins/praetor/skills/dispatching-to-codex/SKILL.md create mode 100755 plugins/praetor/skills/dispatching-to-codex/preflight.sh create mode 100644 plugins/praetor/skills/writing-codex-briefs/SKILL.md diff --git a/README.md b/README.md index e4de615..415f43c 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,7 @@ Install or disable them dynamically with the `/plugin` command — enabling you - [claude-desktop-extension](./plugins/claude-desktop-extension) - [lyra](./plugins/lyra) - [model-context-protocol-mcp-expert](./plugins/model-context-protocol-mcp-expert) +- [praetor](./plugins/praetor) - [problem-solver-specialist](./plugins/problem-solver-specialist) - [studio-coach](./plugins/studio-coach) - [ultrathink](./plugins/ultrathink) diff --git a/plugins/praetor/.claude-plugin/plugin.json b/plugins/praetor/.claude-plugin/plugin.json new file mode 100644 index 0000000..d892461 --- /dev/null +++ b/plugins/praetor/.claude-plugin/plugin.json @@ -0,0 +1,20 @@ +{ + "name": "praetor", + "version": "0.1.0", + "description": "Command the legion, judge the work — Claude plans, Codex executes, and a binding fresh-context judge decides what merges. Zero config.", + "author": { + "name": "luoxianzi" + }, + "license": "MIT", + "keywords": [ + "codex", + "delegation", + "claude-code", + "gpt-5.5", + "verification", + "multi-agent", + "code-review" + ], + "homepage": "https://github.com/luoxianzi/praetor", + "repository": "https://github.com/luoxianzi/praetor" +} \ No newline at end of file diff --git a/plugins/praetor/LICENSE b/plugins/praetor/LICENSE new file mode 100644 index 0000000..5d8a301 --- /dev/null +++ b/plugins/praetor/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 luoxianzi + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/plugins/praetor/README.md b/plugins/praetor/README.md new file mode 100644 index 0000000..b457bc1 --- /dev/null +++ b/plugins/praetor/README.md @@ -0,0 +1,115 @@ +# praetor + +**Claude plans. Codex executes. A judge you can't sweet-talk decides what merges.** + +*The Roman praetor held both imperium — the power to command the legions — and the judgment seat. So does this plugin: command the legion, judge the work.* + +A Claude Code plugin that lets Claude hand grunt work to the [Codex CLI](https://github.com/openai/codex) — **only when you say so** — with acceptance criteria frozen in git before Codex starts, and an independent fresh-context judge whose FAIL cannot be overridden. + +[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE) [![Claude Code plugin](https://img.shields.io/badge/Claude%20Code-plugin-blueviolet)](https://claude.com/claude-code) [![中文说明](https://img.shields.io/badge/文档-中文-red)](README.zh-CN.md) + +--- + +![praetor in action — replay of real runs](docs/assets/demo.gif) + +*Replay of real runs — every number and verdict above comes from the benchmark table below, including the timeout kill. Reproduce it: `vhs docs/assets/demo.tape`.* + +## Why + +- **Your Claude tokens should buy judgment, not grunt work.** Bulk edits, mechanical test-writing, wide read-and-report analysis — these burn context and quota that Claude should spend on design and review. Codex runs them in its own process, on its own quota. +- **Delegation without verification is just hope.** In our live testing, roughly **1 in 3 unattended executor runs failed independent review** — that's the work you'd otherwise have merged. So nothing merges here without a verdict. +- **You stay in charge.** This plugin never auto-dispatches. Claude may *offer* ("this looks Codex-shaped — want me to dispatch it?") — the work only moves when you say yes. + +## Install + +``` +/plugin marketplace add luoxianzi/praetor +/plugin install praetor@praetor +``` + +That's it. **Zero configuration.** Idle footprint: **~313 always-on tokens** — that is the entire cost until the moment you dispatch. If `codex login` works on your machine, dispatch works. No config file, no wizard, no API keys handed to us — the plugin only shells out to your own authenticated Codex CLI. + +Requirements: [Claude Code](https://claude.com/claude-code) + [Codex CLI](https://github.com/openai/codex) (`npm i -g @openai/codex`, then `codex login`). + +## Use + +Say it in plain language, or use the command: + +``` +"send this to codex" · "delegate the refactor to codex" · "交给codex" + +/praetor:delegate migrate all date formatting in src/ from moment to dayjs +``` + +What happens next (the lifecycle): + +``` +you say the word + → preflight (codex installed? logged in? STOP file?) + → worth-it check — if doing it directly is faster, Claude says so first + → throwaway branch codex/ (main is never touched) + → acceptance criteria frozen & committed (before Codex exists) + → self-contained brief → codex exec (gpt-5.5, xhigh effort, sandboxed) + → fresh-context judge runs the frozen checks — PASS or FAIL, binding + → PASS: Claude commits & reports · FAIL: ≤2 retries, then loud takeover + → cleanup + one-line ledger entry +``` + +**Three iron laws — no exceptions:** + +1. **No dispatch without a frozen bar in git.** +2. **No acceptance without the judge.** A FAIL cannot be overridden — not by Claude, not by a persuasive diff. +3. **Max 2 retries, then loud takeover.** Every failure path ends with Claude doing the work and telling you delegation failed. + +Silent failure is treated as the #1 killer of tools like this. It has no path here. + +## Measured, not promised + +Real numbers from repeated local runs are published here before anything else is claimed. Each row: one task class, wall-clock and token cost of *dispatch vs. Claude doing it directly*, and the judge's first-pass verdict rate: + +| Task class | Claude solo | Dispatched | Verdict | +|---|---|---|---| +| Bulk mechanical edit — API rename across 16 files | ~1 min | ~4 min (2.6 min Codex + 1.4 min judge) | **Judge: PASS first try** (12-point review) — merged without reading the diff | +| Tiny task — one-line function | seconds | 1.7 min — and the 1st attempt died at the 4-min timeout | **Don't dispatch small tasks.** The skill says so before you waste the minutes | +| Unplanned bonus: transient stall | — | one 29-min zero-write hang → killed by the timeout law → retry succeeded in 2.6 min | **Loud takeover, never silent failure** — the law fired in real life | + +First published runs — n=1 per arm, synthetic fixtures, one machine; medians replace these as repetitions accumulate. Full honesty: **2 of 4 dispatch attempts stalled** on our test machine and were killed by the hard timeout; both retries succeeded, and the judge passed delivered work on the first review. Wall-clock favors solo on small fixtures — dispatch pays in **quota shift and verified merges**, not raw speed. + +Dispatch has real overhead (branch + freeze + judge). Small tasks are **faster without it** — the skill says so instead of dispatching anyway. + +## How praetor differs + +Other Claude↔Codex bridges exist and are good at what they do. The factual difference: + +| | Who decides to dispatch | What verifies the output | Config required | +|---|---|---|---| +| **praetor** | You, explicitly — never auto | Fresh-context judge; FAIL is binding | None (~313 tokens idle) | +| [codex-plugin-cc](https://github.com/openai/codex-plugin-cc) | You, via /codex commands | You read the result | Codex CLI auth | +| [skill-codex](https://github.com/skills-directory/skill-codex) | Claude, when the skill triggers | You read the result | Codex CLI + model prompts | +| [architect-loop](https://github.com/DanMcInerney/architect-loop) | Automatic within the loop | Gates + review inside the loop | Installer + orchestration setup | + +## Relay / custom model users + +Already pointing your Codex CLI at a relay or another model via `~/.codex/config.toml`? **It just works** — preflight detects a custom provider and respects your config instead of forcing flags. The judge protects quality regardless of the executor: a weaker model means more takeovers, never silently bad merges. + +Officially recommended and tested path: **Codex `gpt-5.5` at `xhigh` effort.** Everything else: supported, not certified. + +Escape hatches (that's all of them): `PRAETOR_MODEL` / `PRAETOR_EFFORT` env vars, and plain language — "don't send this to codex", "stop delegating for now". A `STOP` file in the repo root halts everything. + +## FAQ + +**What leaves my machine, and what do you see?** Nothing comes to us. The plugin shells out to your own authenticated Codex CLI — your keys, your relay, your quota. + +**What happens when Codex fails?** ≤2 retries against the frozen criteria, then Claude loudly takes over and does the work itself. There is no silent-failure path. + +**Can I (or Claude) override a FAIL?** No. That is the product. If you want an overridable judge, [docs/DESIGN.md](docs/DESIGN.md) explains why we won't build one. + +**Will it ever dispatch without me asking?** Never. Claude may offer; the work moves only when you say yes. + +## What's deliberately NOT here + +No config file. No model picker. No concurrency knobs. No background daemon. No dashboards. Retries are fixed at 2 — it's a tested law, not a preference. Every one of these was cut on purpose; see [docs/DESIGN.md](docs/DESIGN.md) before filing the issue. 🙂 + +## License + +MIT diff --git a/plugins/praetor/agents/codex-judge.md b/plugins/praetor/agents/codex-judge.md new file mode 100644 index 0000000..c9ad180 --- /dev/null +++ b/plugins/praetor/agents/codex-judge.md @@ -0,0 +1,29 @@ +--- +name: codex-judge +description: Independent fresh-context judge for Codex-delegated work. Spawned by dispatching-to-codex after Codex finishes. Reviews the uncommitted working tree against the frozen acceptance criteria and returns a binding PASS or FAIL. Never fixes anything; never sees the planner's reasoning. +tools: Bash, Read, Grep, Glob +--- + +You are an INDEPENDENT reviewer. You did NOT plan or perform this work, and you must not be charitable. Your only job is a binding verdict: **PASS** or **FAIL**. + +You are given: a repo path, a branch name (`codex/`), and the frozen bar at `.codex/ACCEPTANCE.md`. + +**Important:** the executor leaves its work UNCOMMITTED in the working tree — the planner commits only after you PASS. Review the working tree (`git diff HEAD`), NOT committed history. The freeze commit that created `.codex/ACCEPTANCE.md` is expected and is not a violation. + +## Do, in order + +1. **Tamper check.** `git -C diff HEAD -- .codex/ACCEPTANCE.md` must be EMPTY. Any change → instant **FAIL: tampered acceptance bar**. +2. **Run every check** in `.codex/ACCEPTANCE.md`, exactly as written, against the working tree. Capture real stdout and exit codes. +3. **Review the diff** (`git -C diff HEAD`) against the GOAL: + - Does it actually achieve the stated GOAL? + - Out-of-scope edits? Deleted or weakened tests? Commented-out assertions? Stubbed/faked results? Silent fallbacks? Any of these → **FAIL**. + +## Verdict rules + +- `PASS` — only if ALL checks exit 0 AND the diff matches the GOAL with no out-of-scope or fake work. +- `FAIL: ` — otherwise. List each failing check with its real output, and each diff concern. +- A check that cannot run (missing command, env error, timeout) is a **FAIL**, not a pass — but say WHY it couldn't run, so the planner can tell broken-work from broken-environment. +- Do NOT fix anything. Do NOT edit files. "The code looks right" never substitutes for a green check. +- Report exactly what you ran and what it printed — evidence, not vibes. + +End your final message with a single line: `VERDICT: PASS` or `VERDICT: FAIL: `. diff --git a/plugins/praetor/commands/delegate.md b/plugins/praetor/commands/delegate.md new file mode 100644 index 0000000..73c3926 --- /dev/null +++ b/plugins/praetor/commands/delegate.md @@ -0,0 +1,10 @@ +--- +description: Delegate a task to Codex with frozen acceptance criteria and a binding independent judge +argument-hint: +--- + +The user explicitly requests delegation of the following task to Codex: + +$ARGUMENTS + +Invoke the `dispatching-to-codex` skill and run its full lifecycle: preflight → worth-it check (say so if doing it directly would be faster, and respect the user's choice) → isolate on a throwaway branch → freeze `.codex/ACCEPTANCE.md` → self-contained brief → dispatch → fresh-context judge → resolve (≤2 retries, loud takeover on failure) → cleanup + ledger. diff --git a/plugins/praetor/skills/dispatching-to-codex/SKILL.md b/plugins/praetor/skills/dispatching-to-codex/SKILL.md new file mode 100644 index 0000000..9b3dfd3 --- /dev/null +++ b/plugins/praetor/skills/dispatching-to-codex/SKILL.md @@ -0,0 +1,49 @@ +--- +name: dispatching-to-codex +description: Use when the user explicitly asks to hand a task to Codex — "use codex", "delegate this", "send it to codex", "交给codex", "派给codex", or the /delegate command. Runs the full dispatch lifecycle with frozen acceptance criteria and a binding independent judge. NEVER auto-dispatch — delegation happens only on the user's word. If a task looks Codex-shaped (bulk mechanical edits, wide read-and-report analysis, test scaffolding), you may offer the option in ONE line; if the user doesn't take it, drop it. +--- + +# Dispatching to Codex + +Claude is the brain; Codex is the hands. You plan, freeze the bar, and judge. Codex executes on its own quota, in its own process, so your context stays small and your tokens go to judgment — not grunt work. + +## Iron Laws (no exceptions) + +1. **NO DISPATCH WITHOUT FROZEN ACCEPTANCE CRITERIA COMMITTED TO GIT.** If you cannot write a concrete pass/fail check, the task is not ready to delegate — do it yourself. +2. **NO ACCEPTANCE WITHOUT A FRESH-CONTEXT JUDGE VERDICT.** A FAIL cannot be overridden — not by you, not by "the diff looks fine". +3. **MAX 2 RETRIES, THEN LOUD TAKEOVER.** Every failure path ends with you doing the work and telling the user delegation failed. Silent success is forbidden. + +## Is it worth dispatching? (say so BEFORE starting) + +Dispatch overhead is real: branch + freeze + judge ≈ minutes of wall-clock and some of both quotas. If you could finish the task faster than the overhead, tell the user plainly: *"This is faster for me to do directly — still want Codex on it?"* Then respect their answer. + +Good dispatches: bulk/repetitive edits, migrations, mechanical test-writing against a clear spec, wide "read everything, report back" analysis, output-heavy runs. +Bad dispatches: design work, subtle debugging, anything ambiguous, tiny tasks. + +## Lifecycle + +0. **Preflight.** Run `${CLAUDE_SKILL_DIR}/preflight.sh`. Not ready → one-line explanation (e.g. "run `codex login` once and I can dispatch"), then do the work yourself. Never nag twice in a session. + +**Analysis-only shortcut:** if the dispatch edits NO files (wide read-and-report, investigation), skip ISOLATE/FREEZE/JUDGE — run `codex exec --sandbox read-only` and review the report yourself. The gate exists for merges, not for reading. Everything below is the write path. + +1. **Isolate.** `git switch -c codex/` — throwaway branch; main is never touched. +2. **Freeze.** Write `.codex/ACCEPTANCE.md`: one-line GOAL + the exact commands that must pass (exit codes, not vibes). Commit it BEFORE Codex exists. See skills/writing-codex-briefs for the format. +3. **Brief + dispatch.** Self-contained brief (Codex sees zero chat history). Stock config: + `codex exec -m gpt-5.5 -c model_reasoning_effort="xhigh" --sandbox workspace-write "" 2>/dev/null` + Model/effort overrides, in priority order: `PRAETOR_MODEL` / `PRAETOR_EFFORT` env vars (use verbatim if set) → preflight reported `config=custom` (active relay provider: DROP the `-m`/`-c` flags, keep `--sandbox`, respect their config) → stock default above. Never `danger-full-access`. Always a hard timeout. Big output → `-o `, never into your context. +4. **Judge.** Spawn a FRESH subagent with `agents/codex-judge.md` + repo path + branch name. Codex's work is UNCOMMITTED — the judge reviews the working tree (`git diff HEAD`), runs every frozen check, checks the bar wasn't moved. Returns PASS or FAIL + evidence. +5. **Resolve.** PASS → you commit (Codex never commits), report in plain language, append the ledger. FAIL → re-brief with `codex exec resume --last ""` (≤2 retries) or take over loudly. +6. **Cleanup.** Merge or delete the branch. Append one line to `.codex/ledger.jsonl`: `{task, model, verdict_history, wall_seconds, dispatched_at}`. + +**Kill switch:** a `STOP` file in the repo root halts all dispatching — check before every dispatch. + +## Red Flags — excuses that mean STOP + +| Excuse | Reality | +|---|---| +| "The diff looks obviously fine, skip the judge" | Obvious diffs are where silent regressions hide. Law 2. | +| "No time to write acceptance criteria" | Then there's no way to know Codex succeeded. Law 1. | +| "One more retry will fix it" | Retry 3+ burns both quotas for a coin flip. Law 3. | +| "Codex said tests pass" | Codex reporting ≠ judge verifying. Only exit codes count. | +| "It's just a small task, skip the branch" | Small tasks on main are how main breaks. Isolate or don't dispatch. | +| "User probably wants this delegated" | Probably ≠ said so. Offer in one line, then drop it. | diff --git a/plugins/praetor/skills/dispatching-to-codex/preflight.sh b/plugins/praetor/skills/dispatching-to-codex/preflight.sh new file mode 100755 index 0000000..356e95b --- /dev/null +++ b/plugins/praetor/skills/dispatching-to-codex/preflight.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# codex-dispatch preflight — is delegation possible right now? +# stdout: one line. exit 0 = ready; non-zero = not ready (message says why). +# Never blocks, never prompts, never writes anything. +set -u + +# Kill switch: a STOP file in the repo root halts all dispatching. +if [ -f "STOP" ]; then + echo "not-ready reason=STOP-file (delete ./STOP to resume dispatching)" + exit 3 +fi + +# Codex CLI on PATH? +if ! command -v codex >/dev/null 2>&1; then + echo "not-ready reason=codex-not-installed (npm i -g @openai/codex)" + exit 1 +fi + +VERSION="$(codex --version 2>/dev/null | head -1 | tr -d '\n')" + +# Logged in? Prefer the CLI's own answer; fall back to auth file presence. +if ! codex login status >/dev/null 2>&1; then + if [ ! -f "${HOME}/.codex/auth.json" ]; then + echo "not-ready reason=not-logged-in (run: codex login)" + exit 2 + fi +fi + +# Dispatch needs git for branch isolation + the frozen acceptance commit. +# (Analysis-only dispatches don't, but write dispatches are the risky path.) +if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + echo "not-ready reason=not-a-git-repo (write dispatches need git; analysis-only is still fine)" + exit 4 +fi + +# Relay / custom provider sniff (read-only): only an ACTIVE top-level +# `model_provider = ...` counts — a defined-but-unactivated [model_providers.x] +# table means the user still runs the official default (real-machine bug, fixed). +CONFIG="stock" +if [ -f "${HOME}/.codex/config.toml" ] \ + && grep -qE '^[[:space:]]*model_provider[[:space:]]*=' "${HOME}/.codex/config.toml" 2>/dev/null; then + CONFIG="custom" +fi + +echo "ready version=${VERSION} config=${CONFIG}" +exit 0 diff --git a/plugins/praetor/skills/writing-codex-briefs/SKILL.md b/plugins/praetor/skills/writing-codex-briefs/SKILL.md new file mode 100644 index 0000000..37907db --- /dev/null +++ b/plugins/praetor/skills/writing-codex-briefs/SKILL.md @@ -0,0 +1,50 @@ +--- +name: writing-codex-briefs +description: Use when composing the task brief and acceptance criteria for a Codex dispatch (invoked from dispatching-to-codex). Covers the self-contained brief format and how to write acceptance checks that actually protect you — weak checks are the sneakiest way a dispatch goes wrong. +--- + +# Writing Codex Briefs + +Codex sees **zero** chat history. The brief is the only channel. Anything not in it does not exist for Codex. + +## The Acceptance File — `.codex/ACCEPTANCE.md` + +Written and committed BEFORE dispatch. This is the frozen bar the judge will enforce. + +``` +GOAL: + +CHECKS (all must pass, exact commands): +- `` exits 0 and prints +- `` exits 0 +- `` exits 0 + +CONSTRAINTS: +- Only edit . Do not modify . +``` + +**Writing checks that protect you** (the judge is only as strong as these): +- Prefer commands with **exit codes** over "the code should look right". +- Include at least one check that **fails before the work is done** (red → green) — a check that passes on the untouched repo proves nothing. +- Name the *specific* tests, not "run the tests" (slow suites make the judge cry wolf; flaky checks are worse than none). +- Vacuous bars ("typecheck passes") let bad work through with a green light. Ask: *could a lazy stub pass these checks?* If yes, tighten them. + +## The Brief + +``` +GOAL: +CONTEXT: +DO: +VERIFY: +REPORT: +DO NOT: commit anything; touch .codex/ACCEPTANCE.md; edit unrelated files; + delete or weaken tests; add silent fallbacks or fake/stubbed results. +``` + +The `DO NOT` block is mandatory and always includes those five items — they are the exact ways executor models paper over failure. + +## Common Mistakes + +- **Context by reference** ("as discussed above") — Codex has no "above". Inline it. +- **Goal drift** — brief GOAL ≠ ACCEPTANCE GOAL means the judge enforces a different bar than Codex aimed at. +- **Kitchen-sink briefs** — one dispatch, one goal. Two unrelated goals = two dispatches.