feat: default-deny authentication middleware (PER-15245)#321
Conversation
The PDP uses an allowlist-of-protected model: every route must individually remember `Depends(enforce_pdp_token)`. The OPAL trigger routers (`/policy-updater/trigger`, `/data-updater/trigger`) are mounted by OpalClient before the PDP wrapper gets control, so they were never gated, and any new route that forgets the dependency fails open. This adds a global ASGI middleware that inverts the model to default-deny: every request must carry a valid PDP token unless its path is explicitly allowlisted. Because it is a global middleware it also gates the routes OPAL mounted before the PDP took over, which a custom APIRoute/APIRouter subclass could not. Design notes: - Pure ASGI (not BaseHTTPMiddleware) to avoid per-request task-group overhead on the hot /allowed path. It runs outside Starlette's ExceptionMiddleware, so it builds the 401 response directly instead of raising, and parses the Authorization header defensively (missing/malformed/invalid -> 401, never a 500 from the unguarded split in enforce_pdp_token). - Two-tier allowlist: tier 1 genuinely-public paths (exact match only, so the gated /healthchecks/opa/* proxy routes are not exposed by a /health prefix), tier 2 routes authenticated by a different mechanism (OPAL listener JWT, container control key) where the middleware steps aside for their own deps. - OPTIONS (CORS preflight) and non-http scopes pass through. - Env-gated rollout mode PDP_AUTH_ENFORCEMENT=audit|enforce (default audit): audit logs would-be-rejected requests and allows them through so the fleet can be instrumented before enforcing; enforce returns 401. Audit logging uses positional loguru args and never logs token material. Wired in at the end of PermitPDP._configure_api_routes so both the production app and the MockPermitPDP test fixture pick it up. Tests run against the real app and cover both modes: fail-closed synthetic route, malformed-header -> 401 (not 500), tier-1 public without token, the /healthchecks/opa/* regression, tier-2 deferral, OPTIONS bypass, and audit-mode logging including that token material is never logged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🔍 Vulnerabilities of
|
| digest | sha256:66dfe2b1b65621365bd8244bcd11a6b4509032eabea80d75b033610638349115 |
| vulnerabilities | |
| platform | linux/amd64 |
| size | 218 MB |
| packages | 247 |
📦 Base Image python:3.10-alpine3.22
| also known as |
|
| digest | sha256:c8f94b3bb77e6ea9015ccd091b7f8aec1b1fcbca95159675235d9a93788797cd |
| vulnerabilities |
Description
Description
Description
Description
Description
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Description
|
There was a problem hiding this comment.
Pull request overview
Implements a default-deny, pure-ASGI authentication middleware to ensure all HTTP routes (including OPAL-mounted trigger routes) require a valid PDP token unless explicitly allowlisted or deferred, with an audit/enforce rollout switch via config.
Changes:
- Added
DefaultDenyAuthMiddlewarewith tiered allowlisting, defensiveAuthorizationparsing, andaudit|enforcebehavior. - Wired the middleware into
PermitPDP._configure_api_routesso it wraps the full app (including OPAL trigger routers). - Added config support (
PDP_AUTH_ENFORCEMENT) and a new integration-style test suite exercising both audit and enforce modes.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| horizon/middleware/default_deny.py | New default-deny ASGI middleware (allowlist + token validation + audit/enforce behavior). |
| horizon/pdp.py | Installs the middleware at the end of app route configuration so it wraps all routes. |
| horizon/config.py | Adds AuthEnforcement enum and AUTH_ENFORCEMENT sidecar config (prefixed as PDP_AUTH_ENFORCEMENT). |
| horizon/tests/test_default_deny_middleware.py | New tests validating middleware behavior against a real app build (including OPAL trigger routes). |
| horizon/middleware/init.py | Package init file (no functional logic). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if schema.strip().lower() != "bearer": | ||
| return False | ||
| # Constant-time compare - this is now the front door for every request. | ||
| return hmac.compare_digest(token.strip(), get_env_api_key()) |
There was a problem hiding this comment.
Addressed in 65a95db. The token check is now wrapped so any comparison/resolution error fails closed to a 401 (never a 500). SystemExit is a BaseException and is deliberately not caught — it signals a fatally-misconfigured PDP that should down the worker rather than be masked as a 401, which matches the existing enforce_pdp_token behavior. In practice the key is resolved and cached at startup (PermitPDP.__init__), so this path is not reachable during request handling.
| # Bypass CORS preflight and allowlisted (public / defer-to-own-auth) routes. | ||
| if method == "OPTIONS" or _is_allowlisted(path): | ||
| await self.app(scope, receive, send) |
There was a problem hiding this comment.
Fixed in 65a95db. The middleware now bypasses only a genuine CORS preflight — OPTIONS carrying Access-Control-Request-Method (exactly what Starlette's CORSMiddleware treats as preflight). A bare/non-preflight OPTIONS to a protected route is now gated, so a future custom OPTIONS handler can't be reached unauthenticated. Covered by test_enforce_cors_preflight_bypassed + test_enforce_non_preflight_options_is_gated.
| # Middleware steps aside; the route's own dependency handles it (here it resolves | ||
| # to 200 because the mock has no OPAL JWT verifier configured). The point is the | ||
| # middleware did NOT emit its 401 - the request reached the route's own auth. | ||
| resp = client.get("/policy-store/config") | ||
| assert resp.status_code == 200 |
There was a problem hiding this comment.
Fixed in 65a95db. This now asserts not _is_middleware_401(resp) (a helper checking 401 + WWW-Authenticate: Bearer + our detail) instead of == 200, so it proves the middleware deferred without coupling to OPAL's current no-verifier policy-store behavior.
| def test_audit_allows_and_logs_would_block(client, audit_logs): | ||
| resp = client.post("/policy-updater/trigger") | ||
| assert resp.status_code == 200 # allowed through in audit mode | ||
|
|
There was a problem hiding this comment.
Fixed in 65a95db — now asserts not _is_middleware_401(resp) instead of == 200. The real assertions (would-block logged with path/method/has_valid_pdp_token) are unchanged.
| def test_audit_valid_token_does_not_log(valid_token, client, audit_logs): | ||
| resp = client.post("/policy-updater/trigger", headers=_bearer(valid_token)) | ||
| assert resp.status_code == 200 | ||
| assert not [r for r in audit_logs if "default-deny audit" in str(r)] |
There was a problem hiding this comment.
Fixed in 65a95db — status check relaxed to not _is_middleware_401(resp); the point of the test (no would-block log emitted when the token is valid) is unchanged.
| def test_audit_never_logs_token_material(client, audit_logs): | ||
| secret = "super-secret-invalid-token-value" | ||
| resp = client.post("/policy-updater/trigger", headers=_bearer(secret)) | ||
| assert resp.status_code == 200 |
There was a problem hiding this comment.
Fixed in 65a95db — now not _is_middleware_401(resp). The key assertions (a would-block WAS logged, but the token value never appears in the log) are unchanged.
| resp = client.post( | ||
| "/policy-updater/trigger", | ||
| headers={"user-agent": "{oops} {0} {malformed"}, | ||
| ) | ||
| assert resp.status_code == 200 |
There was a problem hiding this comment.
Fixed in 65a95db — now not _is_middleware_401(resp). The claim under test (a loguru-hostile User-Agent does not drop the audit line) is unchanged.
zeevmoney
left a comment
There was a problem hiding this comment.
Automated review — PER-15245 (default-deny authentication middleware) [load-bearing]
What this PR does: Adds a pure-ASGI DefaultDenyAuthMiddleware (horizon/middleware/default_deny.py), wired in PermitPDP._configure_api_routes via install_default_deny(app) as the last-added (outermost) middleware, so it sees every route — including the /policy-updater/trigger / /data-updater/trigger routers OPAL mounts before the PDP wrapper runs, which were never gated. It inverts the model to default-deny: every HTTP request needs a valid PDP bearer token unless its path is on an exact public allowlist (/health, /docs, /openapi.json, ...) or a defer-to-own-auth prefix (/policy-store, /callbacks, /opal-server, /_exit). A new PDP_AUTH_ENFORCEMENT config toggles audit (log + allow) vs enforce (401). +507/-0 across 5 files incl. 293 lines of tests.
Verdict: APPROVE. No Postable finding is HIGH or CRITICAL (severity rule: only MEDIUM/LOW postable -> approve). The middleware is well-engineered and the security-critical mechanics verify out:
- Wiring is correct:
install_default_denyis called last in_configure_api_routes, andadd_middlewareprepends, so it runs outermost/first (before CORS and route handlers). - Bypass-resistant allowlist: exact-match tier-1 (a
startswith("/health")trap is explicitly avoided —/healthchecks/opa/readystays gated) and boundary-aware tier-2 prefixes (/callbacksfoodoes NOT match/callbacks). Trailing slash normalized. Token check is constant-time (hmac.compare_digest) and the token value is never logged. - Fail-closed is proven: a synthetic brand-new route with no own auth returns 401 in enforce mode. Tests cover enforce/audit, missing/invalid/wrong-scheme/malformed tokens, public routes, trailing slash, the /health-prefix trap, tier-2 defer (
/_exitreaches its ownenforce_pdp_control_key), OPTIONS preflight, and loguru-injection-safe audit logging that never logs token material. - #321's own tests pass (CI
pytests: 66 passed; the 34 failures are the pre-existing aioresponses/aiohttp-3.14 breakage in other test files, not this PR).
Findings
Postable
| # | Sev | File:Line | Category | Description |
|---|---|---|---|---|
| 1 | MEDIUM | horizon/config.py:325 | Security posture | AUTH_ENFORCEMENT defaults to AUDIT (log + allow-through), so the "default-deny" control is inert on a fresh deploy until PDP_AUTH_ENFORCEMENT=enforce; the OPAL trigger routes it uniquely protects stay open by default. Intentional/documented but contradicts the "default-deny" framing — confirm the rollout end-state. |
| 2 | MEDIUM | horizon/middleware/default_deny.py:98 | Robustness / doc-accuracy | Non-ASCII bearer token -> hmac.compare_digest TypeError -> raw 500 (middleware runs outside ExceptionMiddleware), contradicting the "never a 500" docstring; the malformed-header test only covers ASCII. Fails closed (not a bypass). Compare bytes / guard the check. |
Informational
| # | Sev | Ref | Category | Description |
|---|---|---|---|---|
| I1 | INFO | default_deny.py:98 (Copilot) | Dedup | Copilot already flagged that get_env_api_key() can raise SystemExit on this line (unresolved) — a distinct path from finding 2; the same defensive-guard fix covers both. Not re-raised separately. |
| I2 | LOW | default_deny.py:147 (Copilot) | Hardening | Copilot flagged the unconditional OPTIONS bypass. Low risk today (no sensitive OPTIONS handlers; FastAPI auto-handles preflight), but a future custom OPTIONS handler would be unauthenticated. Acknowledged; not re-raised. |
| I3 | INFO | DEFER_AUTH_PREFIXES | Trust boundary | /policy-store, /callbacks, /opal-server rely on OPAL's own JWT auth, which the middleware cannot verify; /_exit is confirmed gated by enforce_pdp_control_key. A missing own-auth on any deferred OPAL route would be invisible to this middleware — worth periodic verification. |
| I4 | LOW | PUBLIC_EXACT_PATHS | Info disclosure | /openapi.json, /docs, /redoc, /scalar are unauthenticated, so the full route/schema is readable by anyone. Deliberate (docs must load); consider gating docs in production for a hardened PDP. |
| I5 | INFO | cross-PR (#317) | Interaction | In enforce mode this middleware + #317's router-level enforce_pdp_token double-gate the enforcer routes (defense-in-depth, fine). #321 allowlists /health, satisfying the constraint flagged on #317 (its public health router is not broken by the middleware). Token checks are consistent (both read get_env_api_key). |
| I6 | INFO | pdp-tester / docker-scout / security-snyk (CI) | Pre-existing / infra | Red for the usual reasons (k3d breakage, residual base-image CVEs, snyk quota); none caused by this PR. |
Blast radius: Global request path — every HTTP request now passes through this middleware. In enforce mode it newly gates the OPAL trigger routes; in audit mode behavior is unchanged except added WARN logs. No schema/data changes.
Isolation / scope: Well-isolated, all-additive (+507/-0), no unrelated churn — clean separation across a new middleware package, config, wiring, and tests.
| AUTH_ENFORCEMENT: AuthEnforcement = confi.enum( | ||
| "AUTH_ENFORCEMENT", | ||
| AuthEnforcement, | ||
| AuthEnforcement.AUDIT, |
There was a problem hiding this comment.
[MEDIUM] "Default-deny" middleware actually defaults to AUDIT (log + allow-through), not deny
Problem: AUTH_ENFORCEMENT defaults to AuthEnforcement.AUDIT, and in audit mode the middleware logs a would-block WARN and then lets the request through (default_deny.py lines 164-167). So a fresh PDP deploy does NOT reject unauthenticated requests — the "default-deny" control is inert until an operator sets PDP_AUTH_ENFORCEMENT=enforce. In particular the OPAL-mounted trigger routes (/policy-updater/trigger, /data-updater/trigger) — the routes this middleware uniquely protects, since #317's router-level dependency cannot reach them — remain callable unauthenticated by default. (The enforcer routes themselves stay gated unconditionally by their per-route / router-level enforce_pdp_token, so the exposure is limited to the OPAL-inherited routes, and it is not a regression since those were already open.)
Suggestion: Confirm audit-by-default is the intended rollout posture and that flipping the fleet to enforce is tracked as a follow-up (the title/PR describe this as "default-deny," which implies enforce). If a fresh deploy is expected to be protected out of the box, default to ENFORCE and make audit opt-in. Either way, make the intended end-state explicit so audit mode is not left on indefinitely.
There was a problem hiding this comment.
Confirmed intentional — leaving the default as audit. The dedicated PDP fleet reports no HTTP telemetry to the main Datadog org and there is no WAF fallback, so audit mode is the instrument used to find any tokenless legitimate callers before enforcing (0 hits/30d per PER-15245). Audit-mode exposure is limited to the two OPAL-inherited trigger routes and is not a regression (they are open today); every previously-gated route keeps its own dependency regardless of mode. The enforce flip + removal of the audit branch is the tracked end-state under PER-15243, and the install-time "AUDIT MODE ACTIVE" banner keeps a stale-audit fleet greppable.
| if schema.strip().lower() != "bearer": | ||
| return False | ||
| # Constant-time compare - this is now the front door for every request. | ||
| return hmac.compare_digest(token.strip(), get_env_api_key()) |
There was a problem hiding this comment.
[MEDIUM] Non-ASCII bearer token makes hmac.compare_digest raise TypeError -> 500 (a second path past the "never a 500" guarantee)
Problem: hmac.compare_digest(token.strip(), get_env_api_key()) raises TypeError when either argument is a str containing non-ASCII characters. HTTP header values are decoded latin-1, so Authorization: Bearer café (or any byte > 0x7F) yields a non-ASCII token, and the raised TypeError propagates out of this ASGI middleware — which runs outside Starlette's ExceptionMiddleware — as a raw 500. That contradicts the module docstring ("parses the Authorization header defensively so a malformed header yields 401, never a 500"), and test_enforce_malformed_header_is_401_not_500 only exercises ASCII inputs ("garbage", "Bearer", "Bearer ", "", "Bearer a b c"), so it doesn't catch this. It fails closed (deny), so it is not an auth bypass, but it is an unhandled exception on the front door. (This is a distinct path from the get_env_api_key() SystemExit concern already raised on this line.)
Suggestion: Compare bytes so any header value is safe, e.g. hmac.compare_digest(token.strip().encode("utf-8"), get_env_api_key().encode("utf-8")), and/or wrap the token check so any exception maps to "invalid token" (401). Add a non-ASCII case to test_enforce_malformed_header_is_401_not_500.
There was a problem hiding this comment.
Fixed in 65a95db. The compare is now byte-based — hmac.compare_digest(token.strip().encode("utf-8"), get_env_api_key().encode("utf-8")) — and wrapped so any error fails closed to 401, so Authorization: Bearer café no longer raises TypeError → 500. httpx refuses to send non-ASCII header values, so the regression is covered at the unit level in test_has_valid_pdp_token (exercising the latin-1 scope uvicorn produces) rather than via the integration client.
…ONS bypass, robust tests Addresses review feedback from Zeev and Copilot on PR #321: - Compare the bearer token as bytes and guard the check so a non-ASCII token (e.g. "Bearer café") no longer makes hmac.compare_digest raise TypeError -> raw 500; any comparison error now fails closed to 401. SystemExit from get_env_api_key (fatal misconfig) is intentionally not caught, matching the existing enforce_pdp_token behavior. Covered by a non-ASCII unit case. - Bypass only genuine CORS preflight (OPTIONS carrying Access-Control-Request-Method) instead of every OPTIONS request, so a future custom OPTIONS handler stays subject to the default-deny check. - Decouple tests from OPAL's downstream status: tier-2 deferral and audit-mode tests now assert "not the middleware's own 401" via an _is_middleware_401 helper rather than a brittle == 200. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thanks @zeevmoney and the Copilot reviewer — addressed in FixedNon-ASCII bearer token →
Brittle Confirmed, not changed
Acknowledged (informational)
✅ 101 tests pass (52 for this middleware), ruff + format clean. |
…ion-middleware-load-bearing
…ion-middleware-load-bearing
…ion-middleware-load-bearing
zeevmoney
left a comment
There was a problem hiding this comment.
Automated review — PR #321 (default-deny authentication middleware) — load-bearing
Reviewed with python-pro + fastapi-pro (security-focused); findings adversarially verified against OPAL 0.9.6 wiring and the reported production behavior.
Verdict: APPROVE (of the code). Highest confirmed severity is MEDIUM — no CRITICAL/HIGH correctness, auth-bypass, or crash defect. Independently confirmed:
- Middleware ordering is correct: OPAL registers
CORSMiddlewarefirst, soadd_middlewaremakes DefaultDeny the outermost user middleware → it correctly bypasses only genuine CORS preflight and gates bareOPTIONS/HEAD/etc. - No allowlist→protected-route bypass:
..traversal, percent-encoding, trailing-slash, case, and leading//all fail closed. add_middlewareat construction time is safe (lazy stack build); the config enum coerces viaAuthEnforcement(value)so an invalid value fails loud at startup (no silent enforce→audit downgrade); the token compare + defensive header parse never raise.- In
enforcemode the two trigger routes and a synthetic ungated route all return 401 (fail-closed) — the real fix for the reported CVE.
PDP_AUTH_ENFORCEMENT=audit, which logs and allows requests through. So merging + deploying does NOT close the actively-exploited unauthenticated /policy-updater/trigger & /data-updater/trigger vuln until enforce is set on every fleet member. This is the intended, ticket-mandated rollout (no HTTP telemetry / no WAF → instrument first), with a loud startup WARN banner, and is captured in the existing open thread at config.py:325 and tracked under PER-15243. Not re-flagged inline — but do not merge this believing the CVE is closed on deploy.
MEDIUM/LOW inline notes below.
| # Starlette's ExceptionMiddleware. (A SystemExit from get_env_api_key on a | ||
| # fatally-misconfigured PDP is a BaseException, deliberately not caught here, | ||
| # matching the existing enforce_pdp_token behavior.) | ||
| return hmac.compare_digest(token.strip().encode("utf-8"), get_env_api_key().encode("utf-8")) |
There was a problem hiding this comment.
[MEDIUM] Hot-path token check depends on an implicit warm-cache invariant; a cold cache degrades to on-loop blocking I/O or a per-request SystemExit
Problem: _has_valid_pdp_token runs on every non-allowlisted request and calls the sync get_env_api_key() directly on the event loop. get_env_api_key() only short-circuits when the module global _env_api_key is truthy; it is O(1) today only because PermitPDP.__init__ warms it at pdp.py:127 before serving. If that invariant breaks (e.g. an empty/falsy resolved key — ENVIRONMENT level returns sidecar_config.API_KEY verbatim with no non-empty check — or a future construction path that installs the middleware without warming), every request re-enters EnvApiKeyFetcher().get_env_api_key_by_level(), which for PROJECT/ORG levels does synchronous BlockingRequest network I/O with tenacity retry (up to 10 attempts, exponential backoff) on the event loop — freezing concurrent requests — and on failure raises SystemExit, killing the worker per request.
Suggestion: Don't couple the request hot-path to lazy cache warmth. Capture the resolved token at install time / from config, or assert the key is resolved when the middleware is installed so a cold cache fails loudly at startup rather than degrading per-request. Latent for a normally-started PDP, hence MEDIUM.
There was a problem hiding this comment.
Fixed in 2a7c84b. install_default_deny now resolves get_env_api_key() once at install and fails loud (logger.critical + SystemExit(GUNICORN_EXIT_APP)) on an empty/unresolved key — so the per-request check is a guaranteed O(1) cache hit and can never degrade to on-loop blocking network I/O or a per-request SystemExit. This also adds the non-empty assertion nothing did before (pdp.py:127 warms but never checks; the ENVIRONMENT path returns PDP_API_KEY verbatim). Kept the per-request get_env_api_key() read so the token tests keep monkeypatching per-test. New test: test_install_fails_loud_on_empty_key. All 3 app-building test fixtures set a non-empty API_KEY before _configure_api_routes, so none break.
| # OPAL routers, the container control key for ``/_exit``). The middleware steps | ||
| # aside for these; their own route-level dependencies still enforce auth. This | ||
| # is "public to the PDP-token middleware", NOT unauthenticated. | ||
| DEFER_AUTH_PREFIXES: tuple[str, ...] = ( |
There was a problem hiding this comment.
[MEDIUM] Tier-2 DEFER_AUTH_PREFIXES assumes the OPAL verifier is enforcing, with no startup guard or test proving it
Problem: The middleware steps aside for /policy-store, /callbacks, /opal-server, trusting each to enforce its own auth. In OPAL those routes funnel through verify_logged_in, which returns {} (allows through) when verifier.enabled is False (OPAL_AUTH_PUBLIC_KEY unset). If the verifier were ever disabled, these routes would be fully unauthenticated — e.g. /opal-server/... connectivity control, /callbacks CRUD, or /policy-store/config (which can disclose POLICY_STORE_AUTH_TOKEN unless EXCLUDE_POLICY_STORE_SECRETS is set). The PR's own tier-2 test deliberately does not assert a downstream 401 (the mock runs with the verifier disabled), so nothing here proves the deferral is safe.
Production evidence (the reported vuln showed /policy-store/config returning 401 "access token was not provided") indicates the verifier IS enabled in prod — so this is not an active bypass and not a regression — but the middleware relies on it implicitly.
Suggestion: Add a startup assertion that fails closed if any tier-2 prefix is deferred while the OPAL verifier is not enforcing, and add a test that asserts a real 401 from a tier-2 route with the verifier enabled — so "defers to own auth" is verified rather than assumed.
There was a problem hiding this comment.
Fixed in 2a7c84b. install_default_deny now emits a loud, greppable WARNING if app.state.opal_client.verifier is disabled while the tier-2 prefixes defer to it ("must never appear in a managed PDP") — a WARN rather than SystemExit, since dev/local PDPs legitimately run with the verifier off, and it is read via defensive getattr so test fixtures without opal_client stay silent. And per your suggestion, test_tier2_route_rejects_without_token_when_verifier_enabled builds a real enabled JWTVerifier (RSA keypair) and asserts /policy-store/config returns a real 401 that is not _is_middleware_401 — so deferral is now verified, not assumed. Also added tests for the warning firing/not-firing.
| try: | ||
| # Constant-time compare - this is now the front door for every request. | ||
| # Compare as bytes: header values are latin-1 decoded, so a non-ASCII token | ||
| # (e.g. "Bearer café") would make hmac.compare_digest raise TypeError on str |
There was a problem hiding this comment.
[LOW] Comment claims the except guards non-ASCII tokens, but both operands are pre-encoded so compare_digest never raises TypeError
Problem: The comment says a non-ASCII token "would make hmac.compare_digest raise TypeError on str inputs," implying the except Exception -> False is what makes "Bearer café" safe. But the code .encode("utf-8")s both operands first, so compare_digest always receives bytes and never raises TypeError; "Bearer café" returns False via byte inequality, not via the handler. The test_has_valid_pdp_token café case therefore passes for a different reason than the comment states. A maintainer trusting the comment might later drop .encode() (comparing strs) assuming the except still fails closed on non-ASCII — reintroducing the TypeError path. The handler also swallows without logging, so a genuinely unexpected error class becomes a silent fail-closed with no diagnostic.
Suggestion: Correct the comment to state the safety comes from pre-encoding to bytes (not from except), and consider logging the exception type (never the token) inside the handler.
There was a problem hiding this comment.
Fixed in 2a7c84b. Corrected the comment to state the safety comes from pre-encoding both operands to bytes (not the except), with an explicit "do NOT drop the .encode()" warning so a maintainer cannot reintroduce the TypeError path. The except is now documented as a last-resort net and logs the error class (never the token) so an unexpected failure is not silent.
| trailing-slash liveness probe from spuriously getting 401 while staying | ||
| consistent with (never more permissive than) the router's matching. | ||
| """ | ||
| return path.rstrip("/") or "/" |
There was a problem hiding this comment.
[LOW] _normalize_path docstring says "collapse a single trailing slash" but rstrip("/") strips all of them
Problem: The docstring (line 70) says "Collapse a single trailing slash," but path.rstrip("/") strips every trailing slash, so /health// → /health. Not exploitable (stripping trailing slashes can only make an already-public path match or produce a router 404 — it can never turn a protected path into an allowlisted one). Purely a doc/behavior mismatch.
Suggestion: Either match the docstring (removesuffix("/") for a single slash) or update the docstring to say all trailing slashes are stripped.
There was a problem hiding this comment.
Fixed in 2a7c84b. Docstring now says it strips all trailing slashes (matching rstrip("/")), and notes stripping can only make an already-public path match or 404 — never turn a protected path into an allowlisted one.
| @pytest.mark.parametrize("method, path", PROTECTED_NO_OWN_AUTH) | ||
| def test_enforce_allows_valid_token(valid_token, client, method, path): | ||
| resp = getattr(client, method)(path, headers=_bearer(valid_token)) | ||
| assert resp.status_code != 401 |
There was a problem hiding this comment.
[LOW] test_enforce_allows_valid_token uses a loose != 401 assertion that a non-401 middleware block would also satisfy
Problem: With raise_server_exceptions=False, an authenticated request that reaches a route which then 500s offline still passes assert resp.status_code != 401. The test proves "not blocked by the middleware's 401" but would also pass if the middleware started blocking with 403/500. The tier-2 tests already use the precise _is_middleware_401(resp) helper.
Suggestion: Assert not _is_middleware_401(resp) here (and at the analogous != 401 assertions) for consistency and to catch a middleware that blocks with a non-401 status.
There was a problem hiding this comment.
Fixed in 2a7c84b. test_enforce_allows_valid_token and the other != 401 assertions (public-routes, trailing-slash) now use the precise not _is_middleware_401(resp) helper, so a middleware block with any status would fail the test.
… guard, doc/test polish Addresses Zeev's second review on PR #321 (2 MEDIUM, 4 LOW): MEDIUM - hot-path token check no longer depends on lazy cache warmth: install_default_deny now resolves get_env_api_key() once and fails loud (SystemExit) on an empty/unresolved key, so the per-request check is a guaranteed O(1) cache hit and can never degrade to on-loop blocking network I/O or a per-request SystemExit. Kept the per-request read so tests still monkeypatch the expected token per-test. MEDIUM - tier-2 defer no longer silently trusts the OPAL verifier: install now emits a loud, greppable WARNING if app.state.opal_client.verifier is disabled while the tier-2 prefixes defer to it (WARN not SystemExit, since dev PDPs legitimately run with the verifier off; accessed via defensive getattr so test fixtures without opal_client stay silent). Added a test that builds a real ENABLED JWTVerifier and proves /policy-store/config returns a real 401 (not the middleware's) - deferral is now verified, not assumed - plus tests for the warning firing/not-firing and for the empty-key startup failure. LOW - corrected the token-check comment (safety comes from pre-encoding to bytes, not the except) and added error-class logging in the handler; fixed the _normalize_path docstring (strips all trailing slashes); tightened loose != 401 test assertions to _is_middleware_401; updated the stale "OPTIONS bypassed unconditionally" module docstring. 105 tests pass; ruff + format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thanks @zeevmoney — all 6 notes from the second review addressed in MEDIUM
LOW — corrected the token-check comment (safety is pre-encoding to bytes, not the ✅ Full suite green on the integrated branch: 128 passed (the previously-failing aioresponses/aiohttp-3.14 tests now pass via the new |
omer9564
left a comment
There was a problem hiding this comment.
this is way overcomplicated, this should be done using the same dependency that enforces authentication everywhere else.
Backward compatibility isn't really needed because 1 - this API is almost for sure unused by any one of the PDP users, 2 it requires upgrade for the changes to happen, 3 we can send announcement that this is changed so users will know
| description="List of callbacks urls to be ignored even if they are registered in the control plane", | ||
| ) | ||
|
|
||
| AUTH_ENFORCEMENT: AuthEnforcement = confi.enum( |
There was a problem hiding this comment.
what is this config ? why do we need it ?
we can just declare breaking changes and whoever uses the apis unauthenticated and wants to upgrade should know he needs to add authentication
There was a problem hiding this comment.
its more complicated but the idea was to change how authentication is enforced. We can add dependency for each route which is what we had previously and is definitely simpler, but it risks authentication holes when new routes are created. Thats what happened with the /kong endpoint.
Yes I agree the enforce/audit config is overkill, will remove.
There was a problem hiding this comment.
there are ways to enforce it on all routes or "group of routes" but simply setting the dependency on the router/app instead of the route itself.
The current implementation is far from any best practice seen on any project
There was a problem hiding this comment.
In general middleware is something that is applied on ALL routes without any way to exclude routes - forcing you to make the exclusion yourself.
dependency can be applied to routes selectively or group of routes / all routes if needed as a built in feature.
There was a problem hiding this comment.
Please see new redesign using FastAPI dependencies on the exposed Opal Routes. I also added additional tests so ungated routes that are not public will not be exposed as this happened with /kong
Complete the pivot from the custom default-deny ASGI middleware to FastAPI-native, dependency-based authentication. - authentication.py: parse the Authorization header with fastapi.security.HTTPBearer instead of the hand-rolled _is_valid_bearer_token. auto_error=False preserves the PDP's 401 contract (not HTTPBearer's default 403) and lets the control-key 503 take precedence over the header check. Only the constant-time compare_digest against our shared secret remains custom - no framework does that for a shared-secret API key. Registering the bearer scheme also makes the Swagger "Authorize" button work. - Remove horizon/middleware/default_deny.py and its tests; routes are gated via include_router(dependencies=[Depends(enforce_pdp_token)]), with post-hoc injection into the OPAL-mounted trigger routes. - config.py: drop the now-unused AUTH_ENFORCEMENT audit-mode toggle; the dependency approach is always-enforce. - Add auth unit + OPAL-trigger integration tests. Isolate the /health public-route test from the shared stats_manager singleton that test_enforcer_api leaves in a failed state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| # The OPAL trigger routers were mounted by OpalClient before the PDP took over, so | ||
| # the include_router-level dependencies above cannot reach them. Inject the PDP | ||
| # token dependency into those two routes directly. | ||
| _gate_opal_trigger_routes(app) | ||
| # High-signal warning if the OPAL-authenticated routes are left open by a disabled | ||
| # verifier (must never happen in a managed PDP). | ||
| _warn_if_opal_verifier_disabled(self._opal) |
There was a problem hiding this comment.
Good catch — the PR description was stale from the original ASGI-middleware design. That approach was dropped after review feedback in favor of the FastAPI-native model you see here (per-router Depends(enforce_pdp_token) + _gate_opal_trigger_routes for the OPAL-mounted trigger routes). The PR description has been rewritten to match the actual implementation, so it no longer references a middleware.
| # Routes that are genuinely public - EXACT path match only, never a prefix. This is the | ||
| # single source of truth for "no PDP token required", imported by the route-audit test | ||
| # (horizon/tests/test_route_auth_audit.py) and the upcoming PER-15249 CI guard. |
There was a problem hiding this comment.
horizon/tests/test_route_auth_audit.py now exists (added in 18ac4b1) and imports PUBLIC_ROUTE_PATHS: it builds the real app and asserts every non-public route carries a recognized auth gate, failing CI if a new route ships ungated. The comment is now accurate.
| # The OPAL client mounts these trigger routes before PermitPDP gains control, so they | ||
| # cannot be gated by an include_router-level dependency - they are secured post-hoc by | ||
| # _gate_opal_trigger_routes instead. Kept as a frozenset so the route-audit test can | ||
| # assert both are present and authenticated. |
There was a problem hiding this comment.
The route-audit test now exists (18ac4b1): test_opal_trigger_route_is_pdp_gated asserts both OPAL_TRIGGER_ROUTE_PATHS entries are present and gated by enforce_pdp_token — exactly what this comment describes.
| Pure unit tests - no OpalClient, no PDP app build. Header parsing itself is now delegated | ||
| to ``fastapi.security.HTTPBearer`` (and exercised end-to-end with real header strings in | ||
| ``test_opal_trigger_auth.py``), so these tests pin only what is genuinely ours: the | ||
| constant-time token comparison, the 401/503 contract of the two dependencies, and the | ||
| public-route allowlist that the route-audit test relies on. |
There was a problem hiding this comment.
Resolved in 18ac4b1 — test_route_auth_audit.py now exists and relies on PUBLIC_ROUTE_PATHS as the public allowlist, so the docstring reference is accurate.
| records: list[str] = [] | ||
| sink_id = logger.add(lambda message: records.append(message), level="WARNING") | ||
| yield records | ||
| logger.remove(sink_id) |
There was a problem hiding this comment.
Fixed in 6c7e526 — the sink now appends str(message), so records is genuinely list[str] and the substring checks no longer rely on loguru's Message being str-compatible.
Add horizon/tests/test_route_auth_audit.py - the route-audit test that PUBLIC_ROUTE_PATHS' docstring already references but which did not exist. It builds the real app via PermitPDP._configure_api_routes and asserts the structural default-deny guarantee: every mounted route is either explicitly public (in PUBLIC_ROUTE_PATHS) or carries a recognised auth gate (enforce_pdp_token / enforce_pdp_control_key, or OPAL's own JWTAuthenticator / require_listener_token). Gates are matched by dependency presence, not path, so an OPAL route rename can't slip past and an unrecognised mechanism also fails. This turns "default-deny" from a per-router convention into a CI-enforced boundary: a new router, a bare @app.get, or a mounted sub-app added without a dependency now fails this test instead of silently shipping unauthenticated. Includes a negative test proving the audit isn't vacuous, plus a regression guard pinning that both OPAL trigger routes require the PDP token. Also fix a pre-existing ruff I001 import-ordering failure in test_opal_trigger_auth.py so CI can run the audit green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The loguru sink receives a Message (a str subclass); str() at capture time keeps the list genuinely list[str] instead of relying on Message being str-compatible for the substring assertions. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
omer9564
left a comment
There was a problem hiding this comment.
I now understand the problem in adding simple dependency here
LGTM
…efault-deny-authentication-middleware-load-bearing # Conflicts: # horizon/pdp.py
The GitHub runner's floating `stable` toolchain moved to rust 1.97.0, whose clippy flags `for (_, result) in permissions_map.iter()` (for_kv_map), and `-D warnings` makes it fatal — breaking Rust CI repo-wide (main is red too) on code this branch never touched. Apply clippy's own suggested fix: iterate `permissions_map.values()` directly. Semantically identical. Verified: cargo fmt/check/clippy clean, 200/200 pdp-server tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
What & why
Part of the PDP Unauthenticated Operational Endpoints — Authorization Hardening project. Implements PER-15245 — the load-bearing structural fix.
The PDP uses an allowlist-of-protected model: every route has to individually carry
Depends(enforce_pdp_token). Two problems:POST /policy-updater/trigger,POST /data-updater/trigger) are mounted byOpalClientbefore the PDP wrapper gets control, so aninclude_router-level dependency can never reach them — they were reachable unauthenticated.This PR closes (1) directly, moves the enforcer routes to router-level gating, hardens the token check, and closes (2) with a CI-enforced fail-closed audit so the allowlist becomes a real boundary rather than a convention everyone has to remember.
Design
1. Token dependencies (
horizon/authentication.py) now build on FastAPI'sHTTPBearer(auto_error=False):HTTPBearer;auto_error=Falselets us return the PDP's documented 401 (not HTTPBearer's default 403) and letsenforce_pdp_control_key's "control API disabled" 503 take precedence over the header check.authorization.split(" ")).hmac.compare_digeston utf-8 bytes (constant-time; safe for non-ASCII).2. Router-level gating (
horizon/pdp.py,horizon/enforcer/api.py):/healthis split into its own router mounted without auth (k8s/LB liveness probes can't attach the PDP token).dependencies=[Depends(enforce_pdp_token)]at theinclude_routerlevel, so auth runs before per-route deps likenotify_seen_sdk— every enforcer/facts/local/proxy/system-owned route is gated./healthchecks/opa/*correctly stays gated (no/healthprefix trap).3. OPAL-mounted trigger routes (
_gate_opal_trigger_routes): the two trigger routes are mounted by OpalClient before the PDP takes over, so we injectDepends(enforce_pdp_token)into the already-mounted route objects directly (mirroring what FastAPI does atAPIRoute.__init__). Fails loud (SystemExit) if a target route is missing — e.g. an OPAL upgrade renamed it — rather than silently shipping an unauthenticated trigger endpoint.4. OPAL-native (tier-2) routes:
/policy-store,/callbacks,/opal-serverare authenticated by OPAL's own JWT verifier._warn_if_opal_verifier_disabledlogs a high-signal warning whenOPAL_AUTH_PUBLIC_KEYis unset (verifier disabled → those routes effectively open). Expected in local/dev; a managed PDP ships a baked-in public key so the verifier is enabled by default.5. Fail-closed route audit (
horizon/tests/test_route_auth_audit.py) — the structural guarantee. Builds the real app and asserts every mounted route is either explicitly public (inPUBLIC_ROUTE_PATHS, the single source of truth) or carries a recognised auth gate (enforce_pdp_token/enforce_pdp_control_key, or OPAL'sJWTAuthenticator/require_listener_token). Gates are matched by dependency presence, not path, so an OPAL rename can't slip past and an unrecognised mechanism also fails the audit. A new router, a bare@app.get, or a mounted sub-app added without a dependency now fails CI instead of shipping open.Wired in at the end of
PermitPDP._configure_api_routesso both production and the test app pick it up.Tests
test_route_auth_audit.py— the fail-closed guarantee: no non-public route is ungated; a negative test proves the audit isn't vacuous; a mounted sub-app is flagged; regression guard that both OPAL trigger routes require the PDP token.test_opal_trigger_auth.py— integration against the real app: unauth trigger routes → 401; wrong token → 401; malformed header → 401 not 500;/healthpublic;/healthchecks/opa/*stays gated;/versiongated;/_exit→ 503 when control key unset; verifier-disabled warning fires/stays silent appropriately.test_authentication.py— unit tests for the constant-time compare (incl. non-ASCII), the 401/503 contract, and the public-route allowlist.test_enforcer_api.py— every protected enforcer endpoint returns 401 without/with a bad token;/healthpublic;/kong+/nginx_allowed+/authorized_usershappy-path with a valid token.✅ 112 tests pass (106 existing/updated + 6 new audit tests).
ruff check+ruff formatclean.Notes
/policy-store/configsecret-leak and warn-vs-fail-closed behaviour of the OPAL verifier (whenOPAL_AUTH_PUBLIC_KEYis cleared by an override) are pre-existing OPAL behaviours; hardening them to fail-closed for managed PDPs is a tracked follow-up.main. Overlap with PER-15244 PR fix: gate /kong, split /health public, enforce PDP token at router level (PER-15244) #317 is limited topdp.py:_configure_api_routes— trivial rebase if that merges first.cryptography/starlettepins + VEX waivers) and thepdp-testerCI migration from k3d to the Docker runtime — see PER-15358.🤖 Generated with Claude Code