stack/apps/backend/src/app/api/latest
BilalG1 fb9f77843e
Populate ClickHouse analytics tables when seeding preview projects (#1471)
## Summary

In preview-mode deployments (`NEXT_PUBLIC_STACK_IS_PREVIEW=true`) the
project overview dashboard reported **0 total users, 0 monthly active
users, and no live users** on the globe. The internal metrics endpoint
reads user/team totals from the ClickHouse `analytics_internal.*` tables
and "live users" from recent `$token-refresh` events — but those tables
are normally filled by the external-db-sync pipeline, which does not run
in preview deployments, so they were empty.

This makes the preview/demo dummy-data seeder populate ClickHouse
directly:

- **`seedDummyAnalyticsMirrorTables`** — mirrors the seeded users /
teams / contact channels into `analytics_internal.users` / `teams` /
`contact_channels` so the metrics endpoint reports real totals.
- **`seedDummyLiveTokenRefreshEvents`** — emits recent `$token-refresh`
events across distinct countries so the overview globe shows live users.
- **Timestamp clamping** — `bulkRandomTimestampOnDay` and the
page-view/click timestamps are clamped so seeded events are never dated
in the future (future-dated events permanently matched the unbounded
"live users" query).
- **`buildTokenRefreshClickhouseRow`** — shared helper for the
`$token-refresh` ClickHouse row shape.
- **`create-project`** — pre-warms the ClickHouse connection so the
seeding inserts don't pay the cold-start cost.
- **`projects-metrics`** — types the ClickHouse `.json()` results (fixes
a `tsc` error).

Also bundles a seeding performance optimization that skips redundant
idempotency lookups when seeding a brand-new project.

Notes:
- Seeded mirror rows use `sync_sequence_id = 0` so that if the
external-db-sync pipeline ever does run for the project, any real update
supersedes the seeded placeholder under `ReplacingMergeTree` + `FINAL`.
- "Live users" naturally decays out of the ~2-minute window a couple of
minutes after project creation; preview creates a fresh project per
visit, so the initial overview always shows them.

## Test plan

- [x] `pnpm --filter @stackframe/backend typecheck` passes
- [x] `pnpm --filter @stackframe/backend lint` passes
- [x] Created fresh preview projects; overview shows non-zero Total
Users / Monthly Active Users
- [x] `analytics_internal.users` / `teams` / `contact_channels`
populated for the seeded project
- [x] Globe shows 8 live users across 8 distinct countries (verified via
the metrics 2-minute query)
- [x] No future-dated `$token-refresh` events in
`analytics_internal.events`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Refactor**
* Faster preview project creation by pre-warming the analytics database
and reusing the warmed connection.
* Reduced initialization delays and redundant checks when seeding
brand-new projects; creation paths now skip needless probes.
* More efficient, parallelized seeding of teams/users/events with
deterministic handling of token-refresh and session-replay data.
* Safer timestamp generation to avoid future-dated events and deferred
background processing for long-running tasks like payments.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/hexclave/stack-auth/pull/1471?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>
2026-05-22 15:54:24 -07:00
..
(api-keys) Upgrade Prisma to v7 (#1064) 2025-12-26 08:13:34 -08:00
ai/query/[mode] Dashboard: DataGrid refactor + layout (stacked on overview-revamp) (#1338) 2026-04-27 13:50:24 -07:00
analytics/events/batch [Feat]: set flag to disable billing (#1417) 2026-05-06 14:58:06 -07:00
auth Route read-only raw Prisma queries to read replica (#1467) 2026-05-22 11:16:11 -07:00
check-feature-support fix types 2026-01-19 16:51:49 -08:00
check-version update version check alerter 2025-12-11 14:29:44 -08:00
connected-accounts Improved StackAssertionError error logging 2026-05-07 13:29:01 -07:00
contact-channels Upgrade ESLint 2026-02-27 10:58:28 -08:00
data-vault/stores/[id] encrypt neon connection strings, update connections route (#879) 2025-09-09 21:35:07 +00:00
emails stack auth preview mode (#1307) 2026-04-08 16:57:42 -07:00
integrations Handle payout.created and payout.reconciliation_completed Stripe webhook events (#1461) 2026-05-21 14:11:36 -07:00
internal Populate ClickHouse analytics tables when seeding preview projects (#1471) 2026-05-22 15:54:24 -07:00
migration-tests Move /api/v1 to /api/latest 2025-02-05 17:24:43 -08:00
oauth-providers Emit delete tombstone when provider_account_id changes (#1320) 2026-04-10 18:46:04 -07:00
payments Payments app design fixes (#1375) 2026-05-21 14:48:56 -07:00
project-permission-definitions Auto migration (#526) 2025-07-24 02:38:37 +02:00
project-permissions Backend fallback (cloud run) (#1306) 2026-04-11 00:57:37 +00:00
projects Onboarding app & restricted users (#1069) 2026-01-11 17:22:14 -08:00
projects-anonymous-users/[project_id]/.well-known Improved anonymous users (#857) 2025-08-24 11:36:01 -07:00
session-replays/batch [Feat]: set flag to disable billing (#1417) 2026-05-06 14:58:06 -07:00
team-invitations [Revert] team invitation accept email-match check (#1431) 2026-05-13 17:15:11 -07:00
team-member-profiles clickhouse new syncs and verify-data (#1304) 2026-04-08 14:43:22 -07:00
team-memberships Backend fallback (cloud run) (#1306) 2026-04-11 00:57:37 +00:00
team-permission-definitions refactor(dashboard): unify AI chat surfaces on assistant-ui Thread (#1427) 2026-05-15 14:21:00 -07:00
team-permissions Backend fallback (cloud run) (#1306) 2026-04-11 00:57:37 +00:00
teams Data-grid overhaul + session-replays / team-payments dashboard surfaces (#1424) 2026-05-15 14:16:47 -07:00
users refactor(dashboard): unify AI chat surfaces on assistant-ui Thread (#1427) 2026-05-15 14:21:00 -07:00
webhooks/svix-token stack auth preview mode (#1307) 2026-04-08 16:57:42 -07:00
beta-changes.txt Move /api/v1 to /api/latest 2025-02-05 17:24:43 -08:00
changes.txt Move /api/v1 to /api/latest 2025-02-05 17:24:43 -08:00
permission-definitions-pagination.ts refactor(dashboard): unify AI chat surfaces on assistant-ui Thread (#1427) 2026-05-15 14:21:00 -07:00
route.ts Endpoints branching (#659) 2025-04-30 15:39:47 -07:00