Skip to content

fix(compliance): use truthiness form in evaluator_auth skip_if guards#5846

Draft
bokelley wants to merge 2 commits into
mainfrom
claude/issue-5844-evaluator-auth-skip-if
Draft

fix(compliance): use truthiness form in evaluator_auth skip_if guards#5846
bokelley wants to merge 2 commits into
mainfrom
claude/issue-5844-evaluator-auth-skip-if

Conversation

@bokelley

@bokelley bokelley commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Applies a defensive storyboard workaround for an adcp-client runner bug that causes evaluator-specific phases to execute instead of being skipped for creative agents returning creative.supports_evaluator: false.

Root cause (adcp-client runner): When the captured context value is boolean false, the runner's skip_if evaluator mishandles != true strict comparison — evaluating false != true as false instead of true — so the guarded phase runs. The adcp-client bug should be tracked and fixed separately.

Storyboard workaround: Changed all six skip_if guards from "context.supports_evaluator != true" to "!context.supports_evaluator" — semantically equivalent for boolean values and consistent with the truthiness style already used for !context.build_capability_id and !context.creative_feature_id in the compound guards throughout the same file.

The storyboard was correctly authored per protocol semantics; this change is purely defensive during the 3.1 pre-release window.

Also includes initial committed build output for dist/compliance/storyboard-runner-options.{js,d.ts} — compiled from server/src/compliance/storyboard-runner-options.ts via tsc --project server/tsconfig.json. The dist/compliance/ directory is explicitly un-ignored in .gitignore; these files were omitted from the initial source commit.

Closes #5844

Test plan

  • evaluator_auth storyboard phases 2–6 are skipped when agent returns creative.supports_evaluator: false
  • evaluator_auth storyboard phases 2–6 execute when agent returns creative.supports_evaluator: true
  • No != true pattern remains in any skip_if guard in evaluator_auth.yaml
  • Compound guards (|| !context.build_capability_id || !context.creative_feature_id) skip correctly when any operand is falsy
  • Changeset scope check passes (patch for adcontextprotocol)

Generated by Claude Code

claude added 2 commits July 8, 2026 14:44
The adcp-client runner mishandles boolean false in != true comparisons:
context.supports_evaluator evaluates to false but false != true is
evaluated as false instead of true, causing evaluator phases to run
against agents that do not advertise evaluator support.

Replace all six skip_if guards from "context.supports_evaluator != true"
to "!context.supports_evaluator" — semantically equivalent for booleans
and consistent with the existing !context.build_capability_id /
!context.creative_feature_id style already in the compound guards.
The underlying runner bug is tracked separately in adcp-client.

Closes #5844
Generated by tsc --project server/tsconfig.json; outDir is dist/,
dist/compliance/ is explicitly un-ignored in .gitignore. File was
added to server/src/compliance/ without its initial compiled output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

creative/evaluator_auth storyboard steps fail instead of skip when agent declares supports_evaluator: false

2 participants