docs: ADR-0027 feature-usage bitmask in the User-Agent#6500
docs: ADR-0027 feature-usage bitmask in the User-Agent#6500eavanvalkenburg wants to merge 3 commits into
Conversation
Add an ADR, design spec, and per-language bit registry for a lightweight feature-usage signal: a 64-bit mask, emitted as a `(feat=vN.<hex>)` User-Agent comment, stamped per request on first-party (Azure/Foundry) clients only. - docs/decisions/0027-feature-usage-bitmask-user-agent.md — ADR (options-first, with Limitations, Open Questions, and v1->v2 migration) - docs/specs/002-feature-usage-telemetry.md — design spec + implementation plan - docs/specs/feature-usage-bit-registry.md — per-language bit tables + governance Granularity is per package with core broken out per feature (each orchestration pattern and built-in context/history provider). Registries are per language (decoder selects by the language already in the UA). OpenTelemetry emission is deferred (privacy). Docs only; no code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The registry JSON was consolidated into feature-usage-bit-registry.md; point the ADR's two remaining links at the markdown instead of the deleted file (fixes markdown-link-check 404s). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a docs-only proposal describing how Agent Framework SDKs can emit a privacy-scoped, per-process “feature usage” signal via a 64-bit bitmask encoded as a (feat=vN.<hex>) User-Agent comment, including versioning/governance guidance and an implementation plan for Python and .NET.
Changes:
- Adds ADR-0027 documenting the decision (first-party-only UA emission, per-language registries, 64-bit mask, hex encoding, opt-out reuse).
- Adds SPEC-002 detailing the planned mechanism/API surface and per-request stamping approach (httpx hooks + azure-core policy, plus .NET parity notes).
- Adds a per-language bit registry table and governance rules for allocating/maintaining bits.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| docs/decisions/0027-feature-usage-bitmask-user-agent.md | ADR documenting the decision rationale, options, limitations, and open questions for UA-based feature-usage telemetry. |
| docs/specs/002-feature-usage-telemetry.md | Companion design/implementation plan describing accumulator/marking API and per-request first-party-only emission. |
| docs/specs/feature-usage-bit-registry.md | Human-readable per-language bit tables plus encoding/decoding rules, opt-out behavior, and governance. |
| - Same `v<version>.<hex>` comment format ⇒ decoded numbers mean the same thing in | ||
| both SDKs. (.NET's policy was already per-request, so there is no Python/.NET | ||
| timing asymmetry.) |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Code Review
Reviewers: 5 | Confidence: 68%
✓ Correctness
Docs-only PR adding ADR-027, SPEC-002, and a feature-usage bit registry. The design is internally consistent and the hex/bit examples are mathematically correct. The main issue is two broken cross-reference links in the ADR that point to a non-existent JSON file (
../feature-usage-bit-registry.json) when the actual registry file isdocs/specs/feature-usage-bit-registry.md.
✓ Security Reliability
Docs-only ADR and spec for feature-usage bitmask telemetry. The design is well-reasoned about privacy (first-party-only emission, opt-out, coarse granularity, privacy review as blocking precondition). However, the ADR contains two broken links referencing a non-existent
docs/feature-usage-bit-registry.jsonfile — the actual registry is the markdown file atdocs/specs/feature-usage-bit-registry.md. The registry doc itself confirms 'No machine-readable registry file ships today,' contradicting the ADR's JSON link.
✓ Test Coverage
This is a docs-only PR (ADR, design spec, and bit registry) with no code changes. Since no runtime behavior is introduced or modified, there are no test coverage concerns. The spec itself appropriately describes the tests that should accompany the future implementation (step 5 of the implementation plan mentions 'tests for the UA opt-out, first-party scoping, and the live (non-frozen) UA'), which is the correct approach for a design document.
✓ Failure Modes
This is a well-structured docs-only PR (ADR + spec + registry) with no code changes. The only concrete issue found is broken cross-reference links in the ADR: lines 23 and 277 reference a non-existent
../feature-usage-bit-registry.jsonfile, while the actual registry lives at../specs/feature-usage-bit-registry.md. The ADR itself correctly links to the registry on line 278 using the right path, making the inconsistency clear. No runtime failure modes apply since this is documentation only.
✗ Design Approach
The overall direction is reasonable, but the .NET mapping section currently specifies accumulator math that would encode the raw bit index instead of setting that bit, so the emitted mask would decode to the wrong features. I also found two internal contract inconsistencies in the docs: one sentence implies cross-language bit meanings are shared even though the registry explicitly makes them language-specific, and the ADR points readers to a JSON registry file even though this proposal says no machine-readable registry ships in v1.
Flagged Issues
- docs/specs/002-feature-usage-telemetry.md:273-277 defines
FeatureBitas the source-of-truth bit list, but then usesInterlocked.Or(ref _mask, (long)bit), which ORs the ordinal itself into the mask. For example, marking bit 22 would emit0x16instead of setting bit 22, so decoders would silently attribute the request to the wrong features.
Suggestions
- docs/specs/002-feature-usage-telemetry.md:284-285 should not say the decoded numbers mean the same thing in both SDKs; docs/specs/feature-usage-bit-registry.md:31-35 explicitly makes the bit tables language-specific.
- docs/decisions/0027-feature-usage-bitmask-user-agent.md:21-23 should point to the Markdown registry, not
docs/feature-usage-bit-registry.json, because docs/specs/feature-usage-bit-registry.md:208-210 says no machine-readable registry file ships in v1.
Automated review by eavanvalkenburg's agents
|
Flagged issue docs/specs/002-feature-usage-telemetry.md:273-277 defines Source: automated DevFlow PR review |
… decode - ADR option J: the parity test compares the enum against the per-language table in the registry doc, not a (now-removed) JSON file. - Spec .NET mapping: the wire format is shared, but the mask is decoded per-language (select the table via the UA product token) — fixes the "decoded numbers mean the same thing in both SDKs" wording that conflicted with the per-language, non-synchronized bit indexes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Motivation and Context
We can see which Agent Framework packages are installed and that some framework call happened (via the existing
agent-framework-python/{version}User-Agent), but we have no usage-based signal about which features are actually exercised at runtime, nor which are used together (e.g. workflows + MCP + Foundry). This proposes a lightweight, privacy-respecting way to collect that signal for the traffic we can actually read, without standing up new event pipelines.Docs-only (ADR + design spec + registry); no code changes.
Description
Adds:
docs/decisions/0027-feature-usage-bitmask-user-agent.md— ADR (options-first, then decision), including Limitations, Open Questions (privacy review is a blocking precondition), and a v1→v2 migration note.docs/specs/002-feature-usage-telemetry.md— design spec + implementation plan.docs/specs/feature-usage-bit-registry.md— per-language bit tables + governance.Key design choices:
(feat=vN.<hex>)User-Agent comment, stamped per request on first-party (Azure/Foundry) clients only — never sent to third-party providers (no fingerprint leak into logs we can't read). OpenTelemetry emission is deferred, primarily on privacy grounds, left open behind the version prefix.FeatureBitenum is the source of truth (no codegen).AGENT_FRAMEWORK_USER_AGENT_DISABLED.Open items for deciders are called out in the ADR (privacy review, optional dedicated opt-out flag, if/when to add OTel, and .NET maintainers confirming the .NET bit table).
Contribution Checklist