Skip to content

Auditable LB changes, evidence export, and console fixes (F1–F3)#1

Merged
henleda merged 7 commits into
mainfrom
x402-mode2-copilot
Jul 27, 2026
Merged

Auditable LB changes, evidence export, and console fixes (F1–F3)#1
henleda merged 7 commits into
mainfrom
x402-mode2-copilot

Conversation

@henleda

@henleda henleda commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Adds an end-to-end audit trail for every change made to a load balancer, an evidence
export, and two console fixes. Also carries four earlier apply commits that were sitting
unmerged on this branch — see Also in this PR at the bottom.

Why

An audit entry could not answer the questions an auditor asks. Of the 15 actions the
codebase emits, not one apply_* recorded a finding_id — every apply function had it
in scope and dropped it — and nothing recorded the XC namespace, the operator, or which run
a change belonged to. /api/audit existed but nothing in the UI ever called it.

What changed

F3 — auditable LB changes (76e29f9)

  • audit.record stamps run_id / actor / host / tool_version centrally, so no
    mutating path can forget it and a caller cannot override it.
  • finding_id + namespace on all 19 record sites. apply_waf / apply_data_guard /
    apply_api_schema also record kept — "was it rolled back" alone cannot say whether a
    change is still live.
  • ApplyContext carries finding_id, so a rollback failure raised deep in the spine is
    still attributable.
  • New runmeta.py: <out>/run.json provenance manifest (repo commit/branch, per-agent
    models, caps, counts) keyed by a run_id every entry joins back to. Fail-soft — it never
    fails a completed scan.
  • New export.py: normalizes the heterogeneous append-only log into one reviewable row per
    event, and packages a .zip whose manifest SHA-256s every member. vpcopilot export is
    the CLI twin.

F1 + F2 + F3 console (f5017a7)

  • /api/scan was truncating to the last 40 lines (/api/action: 60) — the full transcript
    never reached the client. Both now serve the whole log with a ?since= tail, in a bounded
    scrollable box that sticks to the tail unless you scroll up to read.
  • The HTML report is now reachable from ② Review, plus a stamped ?download=1.
  • ⑤ Retire shows the audit trail before you can export it, so you can check what leaves
    the machine.

Docs + fixture (806717d) — new docs/AUDIT.md, README rewrite, USAGE §7 rewritten (it
described tabs that no longer exist), all five screenshots regenerated.

Verification

  • ruff check src tests demo clean; 186 tests pass, coverage 73% (floor 65%).
  • 30 new tests across test_export.py, test_audit_provenance.py, test_console_*.py.
  • The bundle-verification snippet in docs/AUDIT.md was run verbatim against a real
    bundle: 18 members, 0 hash mismatches.
  • Console changes verified in a real browser, not just unit-tested.

Honest limits (stated in the docs, not just here)

  • The log is written by the process making the change, to a local file. It is not
    tamper-evident
    . The manifest's SHA-256s prove a bundle was not altered after export;
    they say nothing about the log before it.
  • actor is self-asserted — attribution, not authentication.
  • Dry runs are not recorded (nothing changed), and apply_timing only exists for
    console-driven live applies. The bundle manifest states both in-band.
  • Entries written by older builds lack the new fields; the exporter recovers what it can via
    the legacy finding key and the policy index, and leaves cells blank rather than guessing.
  • No backfill of historic audit.log files. Tracked in BACKLOG.md along with bundle
    signing and export --verify.

Review notes

  • The audit record schema change is additive; every existing reader uses .get(), so
    report.py and bench_model are unaffected.
  • demo/out is regenerated. build_demo_out.py now curates actor/host/out_dir so no
    real machine identity or home directory lands in a committed, screenshotted fixture — that
    curation lives in the builder, deliberately not in runmeta, which the audit trail
    depends on being honest.
  • Screenshots were captured against a credential-free .env: with XC creds loaded the hero
    band renders a deep link carrying the tenant hostname and namespace, which would otherwise
    ship in the image. docs/DEMO.md documents that for whoever regenerates them next.

Also in this PR

Four commits that predate this work and were already unmerged on the branch:
4c7aadb per-agent malicious_user + rate_limit via user_identification (A2–A4) ·
eea6c78 api_schema validates headers + query params · 0f506fb --finding threads a
probe id into service-policy apply/refine · 38eeebc data_guard reuses the attached WAF.
Happy to split them onto their own branch if you'd rather review them separately.

🤖 Generated with Claude Code

henleda and others added 7 commits July 23, 2026 21:26
…it (+ --app-firewall/--template flags)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ply/refine

apply --from-scan can now take --finding <id> so the refiner/validator loads the
real probe (out/probes.json keyed by finding_id) instead of falling back to the
Nimbus negative-pay probe. Threaded through cli.apply -> refine_apply_service_policy
(already accepted finding_id) and apply_from_scan -> apply_service_policy.
Enables genuine live validation + self-refine of a generated policy with no scan/ledger.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ation (A2-A4)

Both controls historically keyed abuse on client IP (user_id_client_ip), which
mis-attributes when many agents egress a shared IP behind an edge.

- xc.py: user_identification CRUD (get/create/exists), mirroring the service_policy idiom.
- apply.py: _user_id_ref (tenant-qualified) + _ensure_user_identification (create a
  user_identification keyed on an HTTP header if absent). apply_malicious_user and
  apply_rate_limit take --user-id-header/--user-id-name: when set they create+reference
  the user_identification and drop user_id_client_ip; otherwise behavior is unchanged.
- probe.py: probe_rate_limit takes headers=, so a per-identity limit can be driven from
  one identity and confirmed against another.
- cli.py: apply-maluser and apply-ratelimit expose --user-id-header/--user-id-name;
  apply-ratelimit also wires the existing --behavioral-path and adds repeatable
  --burst-header name=value.

Note (per the Mode-2 plan A4): whether an LB user_identification makes the aggregate
rate_limiter count per-identity is not decidable from source — the two-identity behavioral
burst is the arbiter; if a clean agent is also throttled, a keyed rate_limiter_policy is
needed. Tests cover the header-keyed vs IP-default paths; 139 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An audit entry could not answer the questions an auditor asks. Of the 15 actions the
codebase emits, not one apply_* recorded a finding_id — every apply function had it in
scope and dropped it — and nothing recorded the XC namespace, the operator, or which run
the change belonged to.

- audit.record stamps run_id/actor/host/tool_version centrally, so no mutating path can
  forget it and a caller cannot override it (reserved keys are stripped from **detail).
- finding_id + namespace on all 19 record sites: the 7 apply_* and 3 create_* in apply.py,
  the 5 refine_apply in refiner.py, rollback_failed in engine.py, and retire. apply_waf /
  apply_data_guard / apply_api_schema also record `kept` — "was it rolled back" alone
  cannot say whether a change is still live.
- engine.ApplyContext carries finding_id, so a rollback failure raised deep in the spine is
  still attributable; pr.py writes finding_id alongside the legacy `finding` key.
- runmeta.py: <out>/run.json provenance manifest (repo commit/branch, config, per-agent
  models, caps, counts, actor/host) keyed by a run_id every entry joins back to. Written
  fail-soft from run_pipeline — provenance is evidence, not a gate on a completed scan.
- export.py normalizes the heterogeneous append-only log into one reviewable row per event
  (joined to ledger + findings) and packages a .zip whose manifest SHA-256s every member.
  It keeps EVERY entry on purpose: report.py's impact table filters to before_after or
  behavioral, which would silently drop retire, open_pr, create_* and config-only applies.
- `vpcopilot export [--out] [--output] [--all]` is the CLI twin of the console endpoint.

Additive: every existing reader uses .get(), so report.py and bench_model are unaffected.
Dry runs stay unrecorded by design, and the bundle manifest states that in-band so an
export can never imply more coverage than it has. 30 new tests; 186 pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… Retire (F1-F3)

Three operator-facing gaps, all in the console.

F1 — the scan log was truncated server-side to the last 40 lines (/api/action: 60), so the
full transcript never reached the client at all; and #scanLog was a bare <pre> with no
height cap, which made the global `pre { overflow:auto }` inert — it grew and pushed the
page down. Both endpoints now serve the whole log with a `?since=<n>` incremental tail and
a `log_total` cursor, behind a bounded sink (LOG_MAX) that says in-band when it caps. The
page appends a text node instead of rewriting, so scroll position and selection survive
each poll; it sticks to the bottom only when you were already there, and a "follow" chip
plus a line count appear when you scroll up to read.

F2 — /api/report already rebuilt from the current run dir on every request, so "latest run"
was already true; only the affordance was missing. Added Open HTML report / Download to
Review (in the section markup — renderReview clobbers #reviewBox) and the Download to
Setup, both captioned with the run dir they rebuild from. `?download=1` sends a stamped
attachment via _stamped_name().

F3 — /api/audit existed but nothing in the UI ever called it. Retire gains an Audit trail
card: when, action, the finding that justified it, control + XC object, LB + namespace,
outcome (with the self-heal badge and the 200 -> 403 proof), and who ran it; filterable,
with a per-row raw-JSON expander. The trail is SHOWN before it can be exported, so you can
check what leaves the machine. New read endpoints: /api/audit-events, /api/runs, and
/api/audit-export?scope=run|all streaming the bundle export.py builds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ixture

docs/AUDIT.md (new) is the reference a security/GRC reader gets sent: what is and is not
recorded, all 15 actions with their detail fields, the run.json schema, the normalized
event model and how `outcome` is derived, the bundle contents with a runnable verification
snippet, and worked examples ("which vuln justified this change?", "what is still live?").
It is explicit about the limits — the log is not tamper-evident, `actor` is attribution
rather than authentication, and a bundle is evidence for a human reviewer, not a compliance
certification.

README rewritten around three pillars instead of two; auditability is now first-class
rather than absent. USAGE §7 rewritten — it described Dashboard/Workflow/Ledger/Admin tabs
that no longer exist. DESIGN gains the audit/provenance decisions (identity stamped in one
place; write heterogeneous, normalize at export; fail-soft provenance). QUALITY_PLAN gains
Phase 5 so the F1-F3 tags cited in the source resolve.

Screenshots regenerated — the set predated the Benchmark step and the model switcher, so
all five were stale beyond this change. Captured headless at 1200px/2x against a
credential-free .env: with XC creds loaded the hero renders a deep link carrying the tenant
hostname and namespace, which would have shipped in the image. DEMO.md documents that.

demo/build_demo_out.py writes the finding_id + namespace shape a real run now writes, plus
run.json, so the Retire audit trail is fully populated in the offline dataset instead of
rendering half-blank columns. It also spaces the timestamps (the builder wrote all 13 in
one burst) and curates actor/host/out_dir — no real machine identity or home directory in
a fixture that is committed and screenshotted. That curation lives in the builder, not in
runmeta: the audit trail depends on runmeta being honest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@henleda
henleda merged commit 011f377 into main Jul 27, 2026
4 checks passed
@henleda
henleda deleted the x402-mode2-copilot branch July 27, 2026 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant