stack/apps/backend/src
BilalG1 fb9f77843e
Populate ClickHouse analytics tables when seeding preview projects (#1471)
## Summary

In preview-mode deployments (`NEXT_PUBLIC_STACK_IS_PREVIEW=true`) the
project overview dashboard reported **0 total users, 0 monthly active
users, and no live users** on the globe. The internal metrics endpoint
reads user/team totals from the ClickHouse `analytics_internal.*` tables
and "live users" from recent `$token-refresh` events — but those tables
are normally filled by the external-db-sync pipeline, which does not run
in preview deployments, so they were empty.

This makes the preview/demo dummy-data seeder populate ClickHouse
directly:

- **`seedDummyAnalyticsMirrorTables`** — mirrors the seeded users /
teams / contact channels into `analytics_internal.users` / `teams` /
`contact_channels` so the metrics endpoint reports real totals.
- **`seedDummyLiveTokenRefreshEvents`** — emits recent `$token-refresh`
events across distinct countries so the overview globe shows live users.
- **Timestamp clamping** — `bulkRandomTimestampOnDay` and the
page-view/click timestamps are clamped so seeded events are never dated
in the future (future-dated events permanently matched the unbounded
"live users" query).
- **`buildTokenRefreshClickhouseRow`** — shared helper for the
`$token-refresh` ClickHouse row shape.
- **`create-project`** — pre-warms the ClickHouse connection so the
seeding inserts don't pay the cold-start cost.
- **`projects-metrics`** — types the ClickHouse `.json()` results (fixes
a `tsc` error).

Also bundles a seeding performance optimization that skips redundant
idempotency lookups when seeding a brand-new project.

Notes:
- Seeded mirror rows use `sync_sequence_id = 0` so that if the
external-db-sync pipeline ever does run for the project, any real update
supersedes the seeded placeholder under `ReplacingMergeTree` + `FINAL`.
- "Live users" naturally decays out of the ~2-minute window a couple of
minutes after project creation; preview creates a fresh project per
visit, so the initial overview always shows them.

## Test plan

- [x] `pnpm --filter @stackframe/backend typecheck` passes
- [x] `pnpm --filter @stackframe/backend lint` passes
- [x] Created fresh preview projects; overview shows non-zero Total
Users / Monthly Active Users
- [x] `analytics_internal.users` / `teams` / `contact_channels`
populated for the seeded project
- [x] Globe shows 8 live users across 8 distinct countries (verified via
the metrics 2-minute query)
- [x] No future-dated `$token-refresh` events in
`analytics_internal.events`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Refactor**
* Faster preview project creation by pre-warming the analytics database
and reusing the warmed connection.
* Reduced initialization delays and redundant checks when seeding
brand-new projects; creation paths now skip needless probes.
* More efficient, parallelized seeding of teams/users/events with
deterministic handling of token-refresh and session-replay data.
* Safer timestamp generation to avoid future-dated events and deferred
background processing for long-running tasks like payments.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/hexclave/stack-auth/pull/1471?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>
2026-05-22 15:54:24 -07:00
..
app Populate ClickHouse analytics tables when seeding preview projects (#1471) 2026-05-22 15:54:24 -07:00
auto-migrations Migration tests 2026-02-17 15:58:06 -08:00
lib Populate ClickHouse analytics tables when seeding preview projects (#1471) 2026-05-22 15:54:24 -07:00
oauth Trust hosted domains 2026-05-21 18:23:23 -07:00
private Update implementation 2026-05-12 11:14:30 -07:00
route-handlers Remote dev envs (#1435) 2026-05-19 15:54:18 -07:00
scripts [Feat] new scripts on migrate/seed/init run for internal (#1421) 2026-05-12 16:05:45 -07:00
utils Backend fallback (cloud run) (#1306) 2026-04-11 00:57:37 +00:00
analytics.tsx Fix event capture 2024-08-14 12:49:35 -07:00
globals.d.ts Split backend and dashboard (#83) 2024-06-18 15:49:31 +02:00
instrumentation.ts Onboarding app & restricted users (#1069) 2026-01-11 17:22:14 -08:00
polyfills.tsx Unhandled promise rejections no longer kill the whole server if not in development 2026-05-12 13:14:12 -07:00
prisma-client.tsx Increase replication timeout to 2s 2026-05-21 14:18:39 -07:00
proxy.tsx External db sync (#1036) 2026-02-05 12:04:31 -08:00
s3.tsx session replays (#1187) 2026-02-16 14:15:17 -08:00
smart-router.tsx Move /api/v1 to /api/latest 2025-02-05 17:24:43 -08:00
stack.tsx Rename STACK_SEED_INTERNAL_PROJECT_SECRET_SERVER_KEY to STACK_INTERNAL_PROJECT_SECRET_SERVER_KEY (#1415) 2026-05-06 10:35:08 -07:00