feat(experimentation): add experiment exposures model, warehouse query and summary builder#7740
Conversation
…nd payload mapper
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
github-actions
Bot
added
api
Docker builds report
|
|
/gemini review |
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements the backend infrastructure for tracking and snapshotting experiment exposures from the data warehouse. It introduces the ExperimentExposureSnapshot Django model, ClickHouse queries to bucket exposures by hour or day, and mapper functions to build the exposure payload. The review feedback highlights a performance optimization opportunity in _build_timeseries_points within api/experimentation/mappers.py to reduce its time complexity from O(N^2) to O(N log N) by pre-grouping buckets.
Visual Regression19 screenshots compared. See report for details. |
github-actions
Bot
added
feature
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7740 +/- ##
========================================
Coverage 98.54% 98.55%
========================================
Files 1452 1454 +2
Lines 55821 55989 +168
========================================
+ Hits 55011 55179 +168
Misses 810 810 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…ntities Adopt Flagsmith's product vocabulary (identities) over experimentation jargon (units) across the dataclass, payload and SQL alias, and group buckets before iterating in the mapper helpers.
github-actions
Bot
added
feature
github-actions
Bot
added
docs
github-actions
Bot
added
docs
…aclasses The exposures computation now constructs frozen dataclasses (ExposuresSummary and parts) instead of hand-knitting TypedDict payloads; the JSON shape is defined once, by asdict() at the model boundary. mappers.py folds into services as the pure half of the single compute_exposures_summary entry point.
github-actions
Bot
added
feature
…s-by-variant mapping is_control is derivable from the reserved "control" key and the control-first ordering is presentation — both move to the UI. The variants list becomes a plain mapping, symmetric with the timeseries points.
github-actions
Bot
added
docs
A $multiple sentinel in the variant column collides with any real variant of that name; quarantine now travels as its own query column and ExposureBucket field, so no variant key is reserved.
github-actions
Bot
removed
feature
Unused since the control flag moved client-side, and features.constants already defines and enforces the reservation.
Zaimwa9
left a comment
Zaimwa9
left a comment
There was a problem hiding this comment.
Strong stuff, really neat architecture. I think the alias is the one to resolve and maybe the upper window exclusion in the query
3 participants
docs/if required so people know about the feature. (deferred — flag-gated and not user-visible yet; docs land with the UI.)Changes
Contributes to the experimentation results layer (v0.1: experiment exposures panel). First backend slice — tasks, API and UI follow in stacked PRs.
ExperimentExposures— one row per experiment (OneToOne), updated in place viarecord_refresh(summary, as_of)/record_failure(). Failure stampslast_error_atand preserves the last goodpayload/as_of. Every refresh recomputes the full window (started_at→as_of): identity dedup, multi-variant quarantine and late event delivery make incremental windows unsound.compute_exposures_summary(...)— single entry point: selects bucket granularity (hourly ≤ 72h windows, daily beyond), queries ClickHouse, builds the summary. Query and payload cannot disagree on granularity.$flag_exposureevents. Identities count once, in the bucket of their first exposure (immune to at-least-once duplicate delivery). Identities seen in >1 variant are flagged via an out-of-bandquarantinedcolumn — a$multiplesentinel would collide with a real variant of that name. Bucket boundaries pinned to UTC in the column type, so the driver returns aware datetimes.ExposuresSummary— frozen dataclass tree, serialised once viadataclasses.asdict()at the model boundary. Increments-only payload: per-bucketnew_identitiesper variant plus theexcluded_identitiescount (quarantined identities appear nowhere else). No derivable fields — cumulative series, per-variant totals, headline total, share %, control badge and ordering are all computed client-side, using the reserved"control"key (enforced by the multivariate serializer).How did you test this code?
pytest tests/unit/experimentation/— 247 passed;ruff+mypystrict clean.