Skip to content

Add PostgreSQL dual writes and room metrics#171

Merged
coder13 merged 5 commits into
devfrom
agent/postgres-dual-write
Jul 10, 2026
Merged

Add PostgreSQL dual writes and room metrics#171
coder13 merged 5 commits into
devfrom
agent/postgres-dual-write

Conversation

@coder13

@coder13 coder13 commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a PostgreSQL 17 sidecar and define the app and analytics schemas in Prisma
  • manage schema history with Prisma Migrate and run prisma migrate deploy as a one-shot production Compose service
  • mirror new users, rooms, participant state, attempts, solves, and explicit room deletion into PostgreSQL with deterministic UUIDs and timestamp-guarded upserts
  • store solve penalties in dedicated boolean columns and index user history by creation time plus solve ID for stable cursor pagination
  • mirror only dirty room participants, attempts, and results during live saves instead of replaying the complete room history
  • remove obsolete PostgreSQL attempts when a MongoDB room changes events, while retaining full snapshot writes for explicit backfills
  • collect pseudonymous room lifecycle, solve-count, and authentication-failure events in MongoDB and PostgreSQL with configurable retention
  • enforce Prisma schema validation, migration application, and database drift checks in CI
  • stop client analytics page views from including query strings

Why

This starts ingesting durable solve and analytics data into PostgreSQL while keeping the existing MongoDB-backed application behavior unchanged. Prisma provides the canonical PostgreSQL schema and migration history, while pg remains the small runtime layer for PostgreSQL-specific dual-write upserts.

Incremental writes prevent solve ingestion from becoming quadratic as room history grows. Normalized penalty fields keep solve statistics queryable without JSON extraction, and unchanged same-timestamp retries are true no-ops to reduce unnecessary PostgreSQL churn.

Rollout and privacy

  • MongoDB remains the source of truth and all application reads still use MongoDB.
  • PostgreSQL writes are fail-open during migration: failures are logged without failing the corresponding MongoDB operation.
  • This handles new writes only. A separate batched backfill job is still needed before any cutover.
  • OAuth access tokens are not copied.
  • Analytics identifiers are HMAC-pseudonymized, and metric events exclude names, email addresses, WCA IDs, room secrets, chat, scrambles, and solve times.
  • Mirroring can be disabled with POSTGRES_ENABLED=false; metrics can be disabled independently with METRICS_ENABLED=false.
  • DIRECT_DATABASE_URL can provide a direct migration connection when runtime DATABASE_URL traffic is pooled.

Validation

  • yarn lint
  • yarn test — 45 tests passed
  • yarn build
  • prisma validate
  • Prisma migrations applied to PostgreSQL 17 with zero schema drift
  • live incremental solve and participant dual-write smoke test
  • live room-event attempt replacement smoke test
  • production migration container build and execution
  • production and development Compose validation
  • production dependency audit; only the pre-existing low-severity cookie advisory remains

coder13 added 5 commits July 9, 2026 16:07
Mirror new application data and pseudonymous metrics into PostgreSQL while MongoDB remains the source of truth. Add versioned migrations, Compose services, retention controls, and focused coverage for the migration path.
Describe the migration by its schema responsibility rather than the temporary dual-write rollout strategy. Keep the initial username index compatible with the final schema when migrations are replayed.
Replace the custom migration registry with a canonical Prisma schema and Prisma Migrate history. Run migrate deploy as a one-shot production service and enforce schema validation and drift checks in CI.
Normalize solve penalties and add stable cursor ordering. Mirror only dirty room participants, attempts, and results to avoid replaying room history on every save.
Move shared event and protocol constants to JSON so the Vite client uses ESM while the CommonJS server reads the same data. Import the Material UI theme provider through its supported direct package entry to avoid the optimizer's broken barrel initialization.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant