Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 18 additions & 4 deletions .agents/skills/sync-openapi-spec/references/sync-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This document records what `developers/agent-api-openapi.yaml` keeps from `warp-

1. Drop every tag listed in `EXCLUDED_TAGS`.
2. Drop every path whose tags are a subset of `EXCLUDED_TAGS`, plus every path listed explicitly in `EXCLUDED_PATHS`.
3. Keep every surviving path verbatim, including any `x-internal: true` markers on its operations.
3. Strip any individual operation marked `x-internal: true` from surviving paths. A path whose every HTTP operation is marked internal is dropped entirely.
4. Keep top-level `openapi`, `info`, `servers`, and `components.securitySchemes` verbatim.
5. Keep only the `components.schemas` entries that are reachable from the surviving paths via `$ref` walking (recursive over `allOf`/`oneOf`/`anyOf`/`items`/`additionalProperties`/etc.).

Expand All @@ -20,20 +20,34 @@ Memory stores are gated as `x-internal: true` server-side. They are not part of
### `harness-support`
The `/harness-support/*` endpoints form the worker-to-server contract used by Oz workers (transcripts, snapshots, finish-task signaling, etc.). They are not part of the public API contract — customers should not call them directly. Excluded permanently.

### `factory`
The `/factory/*` endpoints are the internal agent-orchestration API used by the factory pipeline (task management, Linear integrations, syncs, etc.). Not a public API surface; excluded until further notice.

### `memory`
The `/memory_stores` path is tagged `memory` in the server spec (renamed from the former `memory_stores` tag). Same rationale as `memory_stores` above: memory stores are gated and not a stable public API yet.

## Excluded paths (within otherwise-public tags)

These four `agent`-tag paths are excluded individually because the `agent` tag itself remains public:
These paths under otherwise-public tags are excluded individually:

- `/agent/runs/{runId}/handoff/attachments` — handoff plumbing tied to local-to-cloud session handoff.
- `/agent/handoff/upload-snapshot` — handoff plumbing (snapshot upload from a local worker).
- `/agent/conversations/{conversation_id}/fork` — conversation-forking primitive used by the harness, not stable public API.
- `/agent/conversations/{conversationId}/redirect` — internal redirect endpoint.
- `/factory/scorers` — factory-internal scorer creation endpoint; carries the `agent` tag but is not a public API surface.
- `/factory/scorers/{scorer_id}/results` — factory-internal scorer results endpoint; carries the `agent` tag but is not a public API surface.
- `/agent/sessions/{sessionUuid}/redirect` — session redirect plumbing (mirrors the excluded `/agent/conversations/{conversationId}/redirect`).
- `/agent/runs/{runId}/client-events` — internal telemetry endpoint for recording run client events; not a stable public API contract.

If any of these become stable public surfaces, remove them from `EXCLUDED_PATHS` and update this list.

## What we deliberately KEEP that you might expect to be hidden
## What the script filters that you might expect to be public

The script now strips individual operations marked `x-internal: true`, even when the operation lives under an otherwise-public path. This prevents internal inter-agent plumbing (the `/agent/messages/*` and `/agent/events/*` inter-agent messaging and event-polling operations, `/agent/conversations/{conversation_id}/rename`, etc.) from appearing in the public docs reference and `public/openapi.json`. These operations exist server-side but are not part of the stable customer-facing API contract. If an operation is promoted to public, remove its `x-internal: true` marker in `warp-server/public_api/openapi.yaml` and the next sync will include it automatically.
Comment thread
warp-agent-staging[bot] marked this conversation as resolved.

The script also strips parameters and schema properties individually marked `x-internal: true`. This removes internal fields (such as `factory_uid` on `GET /agent/identities` and factory-specific fields on agent request/response schemas) without hiding the entire endpoint or schema. Schemas that were only reachable via stripped properties are also pruned by the existing `$ref` reachability pass.

The script keeps `x-internal: true` operations under public paths. Today this means the `/agent/messages/*` and `/agent/events/*` operations are present in the docs file even though they're flagged `x-internal` in the source. This matches the pre-existing state of `developers/agent-api-openapi.yaml` and the way Scalar already renders the reference. If we want to start stripping `x-internal` operations from the docs spec, change the policy here and update `_should_keep_path`/the operation-level filter in `scripts/sync_openapi.py`.
> **Merge-ordering note (delete once warp-server#13429 lands):** Running `--mode diff` or `--mode apply` against `warp-server develop` will report drift on the `computer_use_enabled` description until warp-server#13429 merges. Always source from that branch (or from `develop` after it merges) to keep the published description in sync.

## Adding a new exclusion

Expand Down
167 changes: 153 additions & 14 deletions .agents/skills/sync-openapi-spec/scripts/sync_openapi.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,15 @@
This script generates the docs subset deterministically:
* tags listed in EXCLUDED_TAGS are removed (and their paths/schemas)
* paths listed in EXCLUDED_PATHS are removed
* surviving paths and operations are kept verbatim, including any
``x-internal: true`` markers
* individual operations marked ``x-internal: true`` are stripped;
a path whose every operation is internal is removed entirely
* parameters marked ``x-internal: true`` are stripped from path-level
and operation-level parameter lists
* schema properties marked ``x-internal: true`` are stripped from every
surviving schema's ``properties`` map (and removed from ``required``)
* components/schemas is pruned to only schemas reachable from the
surviving paths via $ref walking
surviving paths via $ref walking (using the stripped schemas, so
schemas only referenced via stripped properties are also pruned)
* the regenerated spec is validated for unresolved $refs before
being written; apply will refuse to write a broken spec

Expand All @@ -38,10 +43,17 @@

import yaml

# HTTP methods that identify operations within an OpenAPI path item.
_HTTP_METHODS: frozenset[str] = frozenset(
{"get", "put", "post", "delete", "options", "head", "patch", "trace"}
)

# Tags whose paths and tag entry should be removed entirely.
# `memory_stores` is gated as `x-internal` server-side.
# `harness-support` is the worker-to-server contract — not a public API.
EXCLUDED_TAGS: frozenset[str] = frozenset({"memory_stores", "harness-support"})
# `factory` is the internal agent-orchestration API — not a public API.
# `memory` is the memory-stores tag (renamed from `memory_stores`) — not public yet.
EXCLUDED_TAGS: frozenset[str] = frozenset({"memory_stores", "harness-support", "factory", "memory"})
Comment thread
warp-agent-staging[bot] marked this conversation as resolved.

# Specific paths under otherwise-public tags that should be hidden from
# the public API reference. Keep in sync with references/sync-policy.md.
Expand All @@ -51,6 +63,12 @@
"/agent/handoff/upload-snapshot",
"/agent/conversations/{conversation_id}/fork",
"/agent/conversations/{conversationId}/redirect",
# factory-scorer paths are factory-internal but carry the `agent` tag
"/factory/scorers",
"/factory/scorers/{scorer_id}/results",
# internal agent plumbing not part of the stable public API contract
"/agent/sessions/{sessionUuid}/redirect",
"/agent/runs/{runId}/client-events",
}
)

Expand Down Expand Up @@ -133,6 +151,82 @@ def _should_keep_path(path: str, path_item: dict[str, Any]) -> bool:
return True


def _strip_internal_parameters(parameters: Any) -> list[Any]:
"""Return a copy of *parameters* with any ``x-internal: true`` entries removed.

``parameters`` is expected to be an OpenAPI ``parameters`` list (each
entry is either a parameter object or a ``$ref``). Non-list inputs are
returned as-is so callers can unconditionally reassign the key.
"""
if not isinstance(parameters, list):
return parameters
return [p for p in parameters if not (isinstance(p, dict) and p.get("x-internal"))]


def _strip_internal_properties(schema: dict[str, Any]) -> dict[str, Any]:
"""Return a copy of *schema* with any ``x-internal: true`` properties removed.

Also removes the corresponding entries from the ``required`` list so the
schema remains internally consistent. When no properties are stripped the
original dict is returned unchanged (no copy).
"""
props = schema.get("properties")
if not isinstance(props, dict):
return schema
stripped_keys = {
k for k, v in props.items() if isinstance(v, dict) and v.get("x-internal")
}
if not stripped_keys:
return schema
out = dict(schema)
out["properties"] = {k: v for k, v in props.items() if k not in stripped_keys}
if "required" in schema and isinstance(schema["required"], list):
new_required = [r for r in schema["required"] if r not in stripped_keys]
out["required"] = new_required # may be empty — valid OpenAPI
return out


def _filter_internal_operations(path_item: dict[str, Any]) -> dict[str, Any]:
Comment thread
warp-agent-staging[bot] marked this conversation as resolved.
"""Return a copy of *path_item* with any ``x-internal: true`` operations removed.

Also strips ``x-internal: true`` entries from the path-level
``parameters`` list and from each operation's own ``parameters`` list.
Non-operation, non-parameters keys (``summary``, ``description``, etc.)
are preserved verbatim. Returns a dict that may have zero operations if
every HTTP method was marked internal; callers should check
``_has_operations`` before including the result.
"""
out: dict[str, Any] = {}
for key, value in path_item.items():
if key in _HTTP_METHODS:
# Drop this operation if it carries x-internal: true.
if isinstance(value, dict) and value.get("x-internal"):
continue
# Strip x-internal parameters from operation-level parameters list.
if isinstance(value, dict) and "parameters" in value:
stripped = _strip_internal_parameters(value["parameters"])
if stripped != value["parameters"]:
value = dict(value)
if stripped:
value["parameters"] = stripped
else:
del value["parameters"]
out[key] = value
elif key == "parameters":
# Strip x-internal parameters from the path-level parameters list.
stripped = _strip_internal_parameters(value)
if stripped: # omit the key entirely when all parameters are stripped
out[key] = stripped
else:
out[key] = value
return out


def _has_operations(path_item: dict[str, Any]) -> bool:
"""Return True if *path_item* contains at least one HTTP operation."""
return any(key in _HTTP_METHODS for key in path_item)


def _collect_refs(node: Any, refs: set[str]) -> None:
"""Recursively collect every component schema name referenced from ``node``.

Expand Down Expand Up @@ -239,26 +333,38 @@ def transform(source: dict[str, Any]) -> dict[str, Any]:
out["tags"] = out_tags

src_paths = source.get("paths") or {}
kept_paths = {
path: item
for path, item in src_paths.items()
if isinstance(item, dict) and _should_keep_path(path, item)
}
kept_paths: dict[str, Any] = {}
for path, item in src_paths.items():
if not isinstance(item, dict) or not _should_keep_path(path, item):
continue
filtered = _filter_internal_operations(item)
if _has_operations(filtered):
kept_paths[path] = filtered
out["paths"] = kept_paths

seed_refs: set[str] = set()
_collect_refs(kept_paths, seed_refs)

src_components = source.get("components") or {}
src_schemas = src_components.get("schemas") or {}
reachable = _transitive_schemas(seed_refs, src_schemas)

# Preprocess schemas to strip x-internal properties before $ref-walking.
# This ensures schemas that are only referenced from stripped properties
# (e.g. an enum type exclusively used by an internal field) are not pulled
# into the output by the transitive reachability pass.
processed_schemas: dict[str, Any] = {
name: _strip_internal_properties(schema)
for name, schema in src_schemas.items()
}

reachable = _transitive_schemas(seed_refs, processed_schemas)

out_components: dict[str, Any] = {}
for ck, cv in src_components.items():
if ck == "schemas":
out_components["schemas"] = {
name: src_schemas[name]
for name in src_schemas
name: processed_schemas[name]
for name in processed_schemas
if name in reachable
}
else:
Expand Down Expand Up @@ -379,9 +485,18 @@ def _self_test() -> int:
],
"paths": {
"/agent/run": {
# Path-level x-internal parameter (should be stripped).
"parameters": [
{"name": "factory_uid", "in": "query", "x-internal": True},
{"name": "trace", "in": "query"},
],
"post": {
"tags": ["agent"],
"operationId": "runAgent",
# Operation-level x-internal parameter (should be stripped).
"parameters": [
{"name": "_internal", "in": "header", "x-internal": True},
],
"requestBody": {
"content": {
"application/json": {
Expand Down Expand Up @@ -430,7 +545,13 @@ def _self_test() -> int:
"RunReq": {
"type": "object",
"properties": {
"config": {"$ref": "#/components/schemas/Config"}
"config": {"$ref": "#/components/schemas/Config"},
# x-internal property referencing InternalOnly schema;
# both the property and InternalOnly should be pruned.
"_internal_ref": {
"x-internal": True,
"$ref": "#/components/schemas/InternalOnly",
},
},
},
"Config": {
Expand All @@ -452,6 +573,8 @@ def _self_test() -> int:
"RunResp": {"type": "object"},
"MSItem": {"type": "object"}, # only referenced by dropped path
"Followup": {"type": "object"},
# Only reachable via the stripped _internal_ref property.
"InternalOnly": {"type": "object"},
},
},
}
Expand All @@ -464,14 +587,30 @@ def _self_test() -> int:
}, f"unexpected paths: {paths}"

schemas = set(out["components"]["schemas"].keys())
# Config and Mode are reachable transitively (allOf, items)
# Config and Mode are reachable transitively (allOf, items).
# InternalOnly is only referenced from the stripped x-internal property and must be absent.
assert schemas == {"RunReq", "Config", "Mode", "RunResp"}, f"unexpected schemas: {schemas}"

# RunReq should not contain the stripped x-internal property.
run_req_props = set(out["components"]["schemas"]["RunReq"]["properties"].keys())
assert "_internal_ref" not in run_req_props, f"x-internal property survived: {run_req_props}"

tag_names = [t["name"] for t in out.get("tags") or []]
assert tag_names == ["agent"], f"unexpected tags: {tag_names}"

assert out["components"].get("securitySchemes"), "securitySchemes should be preserved"

# Path-level x-internal parameter must be stripped; public one must survive.
run_path_params = out["paths"]["/agent/run"].get("parameters", [])
param_names = [p.get("name") for p in run_path_params if isinstance(p, dict)]
assert "factory_uid" not in param_names, f"x-internal path param survived: {param_names}"
assert "trace" in param_names, f"public path param was stripped: {param_names}"

# Operation-level x-internal parameter must be stripped; the parameters key
# should be absent because all its entries were internal.
run_op_params = out["paths"]["/agent/run"]["post"].get("parameters")
assert run_op_params is None, f"stripped-only op parameters key should be absent: {run_op_params}"

ref_errors = _validate_output(out)
assert not ref_errors, f"unexpected unresolved refs: {ref_errors}"

Expand Down
Loading
Loading