## What
Custom emails / templates / drafts sent through Hexclave's **shared
(development) email server** are no longer blocked with
`RequiresCustomEmailServer`. They are now allowed, but their **subject
and body are wrapped** at send time with a notice that this is a
development email from Hexclave, so unexpected recipients know they can
safely ignore it.
The wrapper only applies to **project-defined content addressed to the
project's own users**. Hexclave's own default-template emails
(verification, password reset, magic link, etc.) and system
notifications (credential-scanning alerts, internal feedback) are sent
**verbatim**.
## How
-
**[send-email/route.tsx](apps/backend/src/app/api/latest/emails/send-email/route.tsx)**
— removed the `RequiresCustomEmailServer` throw that blocked the shared
server.
- **[emails.tsx](apps/backend/src/lib/emails.tsx)** — added
`wrapSharedDevEmail()` (prefixes the subject with `[Hexclave dev email]`
and prepends a notice banner to HTML/text) and
`isCustomEmailForSharedServer(recipient, createdWith, templateId)`.
- **[email-queue-step.tsx](apps/backend/src/lib/email-queue-step.tsx)**
— applies the wrapper at send time, gated on `emailConfig.type ===
"shared"` **and** the email being project-defined custom content.
Applying it at send time reliably wraps both the subject (from
`overrideSubject` or the template's `<Subject>`) and the rendered HTML.
### What counts as "wrap-eligible"
`isCustomEmailForSharedServer` returns true only when **all** hold:
1. the email is addressed to one of the project's own users (recipient
type is not `custom-emails`), **and**
2. it is a draft, a custom template, or raw HTML — i.e. **not** one of
the built-in `DEFAULT_TEMPLATE_IDS`.
Condition (1) exempts Hexclave's own system senders (credential-scanning
revoke, internal feedback) which send raw HTML to bare addresses via
`custom-emails` and would otherwise be mis-classified as project
content. This was a bug caught in review — a leaked-API-key security
alert to a shared-server customer would have been prefixed `[Hexclave
dev email]` with a "you can safely ignore it" banner. The recipient type
is already persisted on the outbox row, so no schema change was needed.
## Tests
- **send-email.test.ts** — replaced the old "400 on shared config" test
with two new tests: (a) a custom email on the shared server is delivered
with the `[Hexclave dev email]` subject prefix + notice banner, and (b)
a **default template** (`sign_in_invitation`) on the shared server is
delivered **verbatim** (no prefix, no banner) — pinning the core safety
contract.
- **js/email.test.ts** — flipped the "throws RequiresCustomEmailServer"
test to assert the send now resolves.
Verified locally against a full stack:
- ✅ `send-email.test.ts` — 18/18
- ✅ `js/email.test.ts` — 12/12
- ✅ `password/send-reset-code.test.ts` — passes (default templates on
shared server stay unwrapped)
## Known limitations (intentional scope)
- **Template CRUD still blocked on the shared server.**
`internal/email-templates` routes still throw
`RequiresCustomEmailServer`, so a shared-server project can send raw
HTML / a default template via the API but cannot create or edit a
*saved* custom template. Sending arbitrary HTML is unaffected; only the
saved-template editor remains gated.
- **A project can send a (project-edited) default template unwrapped**
by calling `send-email` with a `template_id` equal to a built-in
`DEFAULT_TEMPLATE_IDS` value. Low impact (requires a server key, limited
upside), noted for awareness.
## Note: freestyle-mock fix included
[freestyle-mock/Dockerfile](docker/dependencies/freestyle-mock/Dockerfile)
now also accepts `/execute/v3/script`. The `freestyle` SDK bump in #1654
moved to `/v3`, but the mock only served `/v1`+`/v2`, so **all** local
email rendering 404'd (pre-existing `dev` breakage, not from this
feature). The v3 request/response is identical to v2. Happy to split
this into its own PR if preferred.
Out of scope: `emails/email-queue.test.ts` has 2 pre-existing snapshot
failures (`margin:0` vs recorded `margin:0rem`, a
`@react-email/components` version drift in the mock) — those tests use a
custom email server, so this PR's shared-only code path never runs for
them.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Email sending can now proceed when using a shared email server.
* Development-style wrapping is applied to eligible shared-server custom
email content, including HTML notice injection.
* **Bug Fixes**
* Removed the previous blocking “requires custom email server” behavior
for shared-server configurations.
* Default-template emails over the shared server are no longer wrapped.
* **Tests**
* Updated end-to-end and JS email tests to validate both wrapped
custom-email behavior and unwrapped default-template behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
A cleanup pass over recurring production errors triaged from Sentry
(`stackframe-pw` org). The common thread: expected/edge-case conditions
thrown as `HexclaveAssertionError` / `captureError`, so Sentry filed
them as errors (and, for the Stripe ones, Stripe redelivered
indefinitely). Each is handled at the source or logged at the correct
severity.
| Sentry issue | Fix | Risk |
|---|---|---|
|
[STACK-BACKEND-1F5](https://stackframe-pw.sentry.io/issues/STACK-BACKEND-1F5)
— `Unknown stripe webhook type` (`invoice_payment.paid`, `payout.paid`)
| Add both to `ignoredEvents`. They fell through to the throwing `else`
and Stripe redelivered them. (`payout.failed`/`canceled`/`updated`
intentionally left unhandled for now.) | Trivial |
|
[STACK-SERVER-1ZV](https://stackframe-pw.sentry.io/issues/STACK-SERVER-1ZV)
— session-replay `413 Request body too large` | Measure event size in
UTF-8 bytes (was UTF-16 `.length`, which undercounts multibyte content);
drop a single oversized event with a warning instead of shipping a
doomed request | Low |
|
[STACK-BACKEND-140](https://stackframe-pw.sentry.io/issues/STACK-BACKEND-140)
+
[STACK-BACKEND-1F1](https://stackframe-pw.sentry.io/issues/STACK-BACKEND-1F1)
— `Unknown error while sending (test) email` | Classify refused SMTP
connections (`ECONNREFUSED`, surfaced by nodemailer as `code:
'ESOCKET'`) as a typed `CONNECTION_REFUSED` error with a real
user-facing message, instead of falling through to the `UNKNOWN`
catch-all in both the low-level sender and the send-test-email route.
Marked `canRetry` so the queued-email path reschedules with backoff. |
Low |
## Notes
- **Session replay (1ZV):** edited the `packages/template`
source-of-truth; the generated SDK copies are gitignored and regenerated
by CI (`pnpm -w run generate-sdks`). The `TextEncoder` is hoisted out of
the rrweb emit hot path to avoid per-event allocation.
- **Email classification (140/1F1):** the new `CONNECTION_REFUSED`
errorType is additive — other consumers only read `errorType` for
logging, and the send-test-email route only special-cases `UNKNOWN`, so
the new type cleanly bypasses both assertion captures. `canRetry: true`
is safe because the connection is refused before any SMTP exchange (no
message handed off → no duplicate-delivery risk); transient refusals
recover, and a persistent misconfig still fails after
`MAX_SEND_ATTEMPTS`. The one-shot send-test-email path ignores
`canRetry`, so its immediate feedback is unchanged.
## Investigated but intentionally NOT changed here
These were initially included, then reverted so we keep getting Sentry
signal while the root causes are still under investigation:
-
**[STACK-BACKEND-1GM](https://stackframe-pw.sentry.io/issues/STACK-BACKEND-1GM)**
— `Stripe webhook bad customer id`. A subscription-changed event with no
customer (the observed case was a Stripe-CLI test
`payment_intent.succeeded` against a dev-connected account). Skipping is
likely the right long-term fix, but kept the throw for now to keep
observing. Note: in live mode the same path could fire on legitimate
customerless one-time payments / guest checkouts.
-
**[STACK-BACKEND-1CN](https://stackframe-pw.sentry.io/issues/STACK-BACKEND-1CN)**
— `Recovered N stale outgoing request(s)`. This is a self-healing
recovery notice (0 user impact); the underlying cause is the poller
process dying between the claim `UPDATE` and the delete. Kept at
`captureError` to keep collecting data on how often / why it happens.
## Verification
- `typecheck` clean: `@hexclave/backend`, `@hexclave/template`,
`@hexclave/js`, `@hexclave/react`, `@hexclave/next`,
`@hexclave/tanstack-start`
- `eslint` clean on all touched files
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/hexclave/hexclave/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Speed up the Usage page by aggregating metered usage across owned
projects/tenancies with fewer queries and new indexes. Adds E2E tests to
verify team-owned rollups and calendar‑month windows.
- **Performance**
- Added concurrent indexes for `EmailOutbox(tenancyId,
startedSendingAt)` and `SessionReplay(tenancyId, startedAt)`; updated
Prisma schema.
- Group tenancies by (DB client, schema) and run one SQL per group that
counts both emails and session replays; uses `mapWithConcurrency` from
`@hexclave/shared` (concurrency 4, aborts on first error).
- Added helpers `getOwnedProjectAndTenancyIdsForBillingTeam` and
`getNonAnonymousUserCountForTenancies`; made `mapWithConcurrency`
null‑safe with bounds checks.
- **Tests**
- Added E2E tests for the internal plan-usage endpoint covering
team-owned rollups, calendar‑month boundaries, and zero‑usage cases.
- Added unit tests for ownership scope resolution and non‑anonymous user
counting.
<sup>Written for commit 5d6098006c.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1650?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Performance Improvements**
* Improved plan usage rollups by aggregating metered emails and session
replays together across an owned scope.
* Added database indexes to speed up time-window metering lookups for
email outbox and session replays.
* **Tests**
* Extended unit tests for billing-team entitlement aggregation and
non-anonymous user counting.
* Added end-to-end coverage for the internal plan-usage endpoint,
including seeded scenarios and period validation.
* **Refactor**
* Reworked entitlement and usage calculations to reuse shared logic for
more consistent results.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: armaan <armaan@stack-auth.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
### Context
Stripe recommends acking webhook events ASAP with a 200. Stripe also
recommends employing event idempotency on your end. By responding
quickly, you prevent stripe from thinking the webhook failed and
retrying the event. Retrying the event in the past used to be
responsible for people getting multiple payment receipt emails. Note
that even in the case where an event processing genuinely fails, we have
a new table to let us recover from it.
Currently, recovery will be manual, but since it will be logged to
sentry we will be notified.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Quick-ack Stripe webhooks with 200 and add atomic idempotency to stop
duplicate processing and emails. Events are persisted and processed in
the background with clear status and error tracking.
- **New Features**
- Persist each webhook in `StripeWebhookEvent` keyed by `event.id` with
full `payload` and `stripeAccountId` for recovery.
- Return 200 immediately; process in the background and track status as
`PENDING`, `PROCESSED`, or `FAILED`.
- Single-flight claim deduplicates redeliveries while `PENDING` and
after `PROCESSED`; only `FAILED` events reprocess on redelivery.
- Store `lastError` on failures; unknown webhook types ack with 200 and
are handled asynchronously.
- Webhook response includes `deduplicated: true` when a redelivery is
skipped.
- **Migration**
- Run Prisma migrations to create the `StripeWebhookEvent` table, enum,
and unique index on `stripeEventId`.
<sup>Written for commit 59456a36e8.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1664?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added persistent, idempotent Stripe webhook handling with event-level
deduplication keyed by the webhook event id.
* Webhooks are acknowledged immediately and processed asynchronously,
with automatic retry capability for failed events.
* **Bug Fixes**
* Reduced duplicate side effects from redeliveries (including preventing
repeated receipt emails) by ensuring only one successful processing per
event.
* **Tests**
* Updated and expanded integration and end-to-end coverage for
asynchronous processing, deduplication, and failure recovery behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
Speeds up project onboarding by consolidating API calls and adding
prefetching:
- Single PATCH endpoint for saving onboarding status + state together
- Background config save on welcome step (with proper retry/error
handling)
- Prefetch email themes and Stripe info for upcoming steps
- Suspense-based skeleton loaders instead of full-page spinners
- Extracted shared types and lightweight step components
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Unified onboarding progress saving for the new project flow,
persisting status plus optional onboarding state in a single update
path.
* **Improvements**
* Onboarding wizard now derives status/state from owned project data and
removes extra internal fetching/loading gates.
* Added skeleton/Suspense loading for email theme and payments steps,
with more reliable “final config”/completion sequencing.
* Admin project data now includes onboarding state; Stripe account info
uses cached retrieval.
* Backend avoids source-of-truth override changes during metadata-only
updates.
* **Tests**
* Updated onboarding wizard tests and added end-to-end coverage for
updating onboarding status/state together.
* **Documentation**
* Added the “clickmaps” app icon to docs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: armaan <armaan@stack-auth.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
## Summary
Completes the env-var side of the Hexclave rebrand: every
`STACK_*`-prefixed variable (including `NEXT_PUBLIC_STACK_*` and
`VITE_STACK_*`) is renamed to `HEXCLAVE_*` across all checked-in `.env`,
`.env.development`, and `.env.example` files (30 files, ~135 keys).
Legacy `STACK_*` names keep working everywhere via dual-read, so
**existing deployments, `.env.local` files, and self-hosted setups need
no immediate migration**.
## How legacy names keep working
- **Server code** already resolves `HEXCLAVE_*` first with `STACK_*`
fallback via `getEnvVariable`. Direct `process.env.STACK_X` readers fed
by the renamed files (prisma seed, e2e tests/helpers, internal-tool
scripts, examples, `prisma.config.ts`) now read `HEXCLAVE_X || STACK_X`.
- **Client code** (Next.js build-time inlining) uses literal dual-read
expressions; the dashboard's `_inlineEnvVars` already had them.
- **Docker/self-hosting**: `docker/server/entrypoint.sh` (shared by the
server and local-emulator images) gets a generic two-way
`HEXCLAVE_`↔`STACK_` env mirror — runs at startup and again before
sentinel replacement — replacing the previous URL-trio-only mirror.
Operators can use either prefix.
## The empty-placeholder trap (`||` vs `??`)
The checked-in templates define empty placeholders (`HEXCLAVE_X=#
comment` parses to `""` via dotenv). With `?? `-based fallbacks, that
empty string would silently shadow a real value under the legacy name —
including legacy vars set in Vercel/CI env at build time, since the
tracked `.env` is present during builds. All fallback chains therefore
treat empty-as-unset (`||`):
- `getEnvVariable` and `getProcessEnv` in `packages/shared`
- the dashboard/docs/example literal dual-reads
- the generated SDK env getters (via
`packages/template/scripts/generate-env.ts`; the generated
`src/generated/env.ts` files are gitignored and regenerate at build)
## Other notable changes
- Tests that override env now set the canonical `HEXCLAVE_*` name (it
wins over `STACK_*`): e2e `cross-domain-auth`, backend
`internal-feedback-emails` in-source test.
- e2e `helpers.ts` port-prefix expansion loop also matches the
`HEXCLAVE_` prefixes.
- `docker/local-emulator/generate-env-development.mjs` reads source keys
canonically (legacy fallback) and emits canonical keys; regenerated
output matches.
- `rotate-secrets.sh` falls back to
`HEXCLAVE_DATABASE_CONNECTION_STRING`.
- Docs code snippets (`docs/code-examples`) renamed outright to
canonical names, consistent with #1571.
- OAuth callback `console.warn` in `packages/template/src/lib/auth.ts`
now says Hexclave.
## Migration note for the team
Local `.env.local` files with legacy `STACK_*` overrides keep working
**unless** the override targets a var that `.env.development` now sets
to a real (non-empty) `HEXCLAVE_*` value — the canonical name wins over
file precedence. Rename those keys in your `.env.local` once.
## Verification
- `typecheck` + `lint` pass on every touched package (shared, backend,
dashboard, e2e, internal-tool, cli, docs, template). Pre-existing
failures on dev (`admin-app-impl.ts` typecheck, dashboard metrics-page
errors) are unchanged (identical error counts with/without this change).
- `getEnvVariable`/`getProcessEnv` fallback semantics smoke-tested
directly (empty-HEXCLAVE → legacy fallback, HEXCLAVE wins when set,
defaults intact).
- `internal-feedback-emails` in-source vitest passes; emulator env
generator `--check` passes; `bash -n` on touched shell scripts.
- Two independent review agents audited the diff for correctness bugs
and coverage gaps; all confirmed findings are fixed in the third commit.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Renamed all `STACK_*` env vars (including
`NEXT_PUBLIC_STACK_*`/`VITE_STACK_*`) to `HEXCLAVE_*` across env
templates and code, with dual‑read that treats empty as unset, detects
conflicts, ignores post‑build sentinels, and falls back to legacy names.
All GitHub Actions now use `HEXCLAVE_*`; local‑emulator e2e is fixed by
setting `NEXT_PUBLIC_HEXCLAVE_IS_LOCAL_EMULATOR` in CI.
- **Refactors**
- Added conflict‑aware dual‑read helpers (prefer `HEXCLAVE_*`,
empty‑as‑unset, ignore post‑build sentinels, preserve empty passthrough)
and used them across `packages/shared` (resolver + tests),
`apps/dashboard` inline/public envs (with tests), `apps/backend` Prisma
config/seed and vitest (accept both prefixes), `packages/cli`
(API/Dashboard URLs, project ID, `HEXCLAVE_EMULATOR_HOME`; tests),
Docker (`entrypoint.sh` mirroring + `rotate-secrets.sh` DB URL),
docs/components (`docs/src/lib/env.ts`), and examples; hosted/Vite apps
now error if both spellings differ.
- Port‑prefix expansion includes `HEXCLAVE_*`; backend tests use a new
helper to resolve DB connection strings; Prisma prefers
`HEXCLAVE_DATABASE_CONNECTION_STRING` with legacy fallback.
- Generated SDK env getters use plain `HEXCLAVE_*` || `STACK_*` (no
conflict throw); dashboard inline resolver preserves empty/sentinel
passthrough to avoid build failures; docs/examples include dual‑read
utilities.
- Tests now stub canonical `HEXCLAVE_*` flags (e.g., plan limits, bot
challenge, OAuth tokens, hosted handler) to avoid shadowing/conflict
with committed defaults.
- **Migration**
- No immediate action; legacy `STACK_*` names still work.
- If both names are set with different values, builds/scripts error. Set
only `HEXCLAVE_*` or make both equal.
- SDK consumers won’t see conflict throws; update env names to
`HEXCLAVE_*` over time.
<sup>Written for commit 7539fb9fbf.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1588?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Migrated environment variable names from the legacy `STACK_*` prefix
to the new `HEXCLAVE_*` prefix across backend, dashboard, tooling,
Docker, and examples.
* Updated environment/config resolution to prefer `HEXCLAVE_*`, treat
empty strings as unset, and detect conflicts when both `STACK_*` and
`HEXCLAVE_*` are set to different values.
* Updated local emulator, server startup, and env-generation workflows
to use the new names (with legacy fallback where applicable).
* **Documentation**
* Updated docs and code examples to reference `HEXCLAVE_*` variables.
* **Tests**
* Refreshed unit and e2e coverage to validate dual-read behavior,
conflict detection, and empty-value handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What
Performance pass on the internal **platform-analytics** route. All 17
ClickHouse queries fire in a single `Promise.all` on the shared
`stackframe` admin user, which is subject to a **9 GB per-user** memory
cap — so the worst case is the *sum* of per-query peaks, not the max.
Benchmarked at 10k projects / 1M users / 50M events (power-law, top
project ≈100k users), the sum of peaks was ~6.7 GiB. This PR brings it
down to ~3.8 GiB.
## Changes
**ClickHouse — `sipHash64(user_id)` as the distinct key** (exact,
verified byte-identical):
| query | peak mem | Δ |
|---|---|---|
| `dauSeries` | 949 → 373 MiB | −61% |
| `mauProjects` | 715 → 313 MiB | −56% |
| `activeByProject` | 635 → 374 MiB | −41% |
| `sparkByProject` | 1165 → 809 MiB | −31% |
A 64-bit hash has negligible collision probability over 1M users; the
benchmark confirmed identical output. (Same trick already used in the
internal-metrics MAU query.)
**ClickHouse — sample the activity split**
(`new`/`retained`/`reactivated`):
The split was the single heaviest query (~1.3 GiB) — its cost is a
window function over ~25.8M `(user, day)` rows plus an all-history scan,
which `sipHash` alone barely helped (−7%). It now uses **consistent
1-in-4 user sampling** (same `cityHash64(user_id) % 4` bucket applied to
both subqueries so each sampled user's full activity sequence is
preserved; counts scaled ×4):
- **317 MiB (−78%)** peak memory, **~0.4% mean error** (max 1.4% on the
smallest day) vs the exact result.
This is an **approximation** — the dashboard "Growth quality" chart now
notes it (`subtitle: "… · sampled estimate (~0.4%)"`).
`ACTIVITY_SPLIT_SAMPLE` is a single constant in the route; set it to `1`
to go back to exact.
## What I tried that did NOT make the cut (documented in the harnesses)
- `country` — peak memory is dominated by the per-user `argMax(country,
event_at)` payload, not the key, so hashing does nothing. Left
exact/unchanged.
- PG `authMethods` / `email` — with the production composite PK indexes
the original plans are already best; correlated-subquery / anti-join
rewrites were far worse. No PG query changes in this PR.
## Benchmark harnesses (added)
- `apps/backend/scripts/benchmark-platform-analytics.ts` — full-route
baseline (per-query time/memory/rows).
- `apps/backend/scripts/optimize-platform-analytics.ts` — sipHash & PG
variant comparison with byte-equality checks.
- `apps/backend/scripts/optimize-split.ts` — exact vs sampled split
variants with accuracy measurement.
They seed isolated `bench_pa` databases (server-side, auto-cleaned) and
read `system.query_log` / `EXPLAIN (ANALYZE, BUFFERS)`. Run e.g.:
`pnpm --filter @hexclave/backend run with-env:dev tsx
scripts/optimize-split.ts`
## Testing
- Backend `typecheck` passes. (Dashboard has pre-existing typecheck
errors on the base branch in unrelated files — auth-methods,
team-analytics, user-emails, RDE config — not touched here.)
- All exact rewrites verified byte-identical to the originals by the
harnesses; the sampled split measured at ~0.4% mean error.
Numbers are local warm-cache (relative shape, not production latency).
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Cuts worst-case ClickHouse memory for the internal platform analytics
route by switching to hashed distinct keys and sampling the heaviest
query. On a 10k projects / 1M users / 50M events benchmark, the sum of
per-query peaks drops from ~6.7 GiB to ~3.8 GiB with exact results (or
~0.4% error on the sampled chart).
- **Performance**
- Use sipHash64(user_id) as the distinct key in uniqExact/uniqExactIf
for DAU series, MAU/projects, active-by-project, and sparkline. Exact
results (verified). Peak memory down 31–61% per query.
- Sample the new/retained/reactivated split at 1-in-4 users (consistent
`cityHash64` bucket across subqueries, counts ×4). Peak memory ~−78%
(~1.3 GiB → ~0.3 GiB) with ~0.4% mean error. Toggle via
`ACTIVITY_SPLIT_SAMPLE` (set to 4; set to 1 for exact). Dashboard
subtitle now notes “sampled estimate (~0.4%).”
- Added local harnesses to seed isolated data and measure
time/memory/equality:
`apps/backend/scripts/internal-analytics/benchmark-platform-analytics.ts`,
`optimize-platform-analytics.ts`, `optimize-split.ts`.
<sup>Written for commit 60ccf1a06f.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1632?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Updates
* **Improvements**
* Enhanced platform analytics calculations for more consistent and
efficient user counting across key performance indicators (DAU, MAU,
per-project metrics).
* Updated the Growth Quality chart to indicate that user counts
represent sampled estimates with approximately 0.4% margin of error for
improved performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mantrakp04 <mantrakp@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mantra <mantra@stack-auth.com>
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/hexclave/hexclave/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Add platform-wide analytics to the internal dashboard with a secure
backend route and a new page to visualize cross-project metrics. Only
available when viewing the `internal` project and gated by platform
admin access.
- **New Features**
- Backend: add `/api/latest/internal/platform-analytics` aggregating
metrics across all projects via ClickHouse; protected by
`ensurePlatformAdmin`.
- Dashboard: add `/projects/[projectId]/platform-analytics` page with
charts; sidebar entry appears only when `projectId === "internal"`.
- **Bug Fixes**
- Correctness: add `branch_id` filters to all event queries and project
aggregates; exclude the `internal` project from ClickHouse aggregates;
validate MRR quantity.
- Metrics/UI: feature adoption uses `total_projects` from the API and
clamps both chart and label to 0–100%; remove unreachable
`revenue_growth` sort key.
- Safety/Tests: use `Map` for country aggregation; add unit tests for
`ensurePlatformAdmin`/`isPlatformAdmin`; switch tests to inline
snapshots and document the `as-any` cast.
<sup>Written for commit 3c803a8915.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1626?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Platform Analytics dashboard for internal projects with
interactive 7/30-day range charts, KPI tiles, and visual breakdowns
(growth, country, sign-in method, user mix), plus email health,
dead-click insights, a searchable project leaderboard, and feature
adoption.
* Introduced an internal analytics API providing rolling-window
comparisons and structured metrics for dashboard rendering.
* **Bug Fixes**
* Strengthened access control with platform-admin authorization for
analytics access.
* **Tests**
* Added coverage for platform-admin authorization behavior.
* **Chores**
* Updated Next.js to 16.2.9 across applications.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mantra <mantra@stack-auth.com>
## Summary
- Fetches the canonical Hexclave skill from `https://skill.hexclave.com`
when the backend AI route is invoked through MCP `ask_hexclave`
- Appends that skill content to the spawned docs agent's system context
before generation
- Adds focused tests for non-Ask-Hexclave no-op behavior, successful
skill embedding, and loud fetch failure
## Why
The public MCP server exposes the skill as a separate resource/prompt,
but the backend docs agent spawned by `ask_hexclave` only saw the user's
question. That meant clients had to correctly load the skill themselves,
and the server-side answer quality could miss the canonical
setup/context.
## Validation
- `pnpm -C apps/backend exec eslint
'src/app/api/latest/ai/query/[mode]/route.ts'
src/lib/ai/mcp-skill-context.ts src/lib/ai/mcp-skill-context.test.ts`
- `pnpm exec vitest run
apps/backend/src/lib/ai/mcp-skill-context.test.ts --config /dev/null
--environment node`
- `pnpm exec tsc --noEmit --target es2022 --module esnext
--moduleResolution bundler --lib es2022,dom --types vitest
apps/backend/src/lib/ai/mcp-skill-context.ts
apps/backend/src/lib/ai/mcp-skill-context.test.ts`
## Notes
- Normal backend Vitest/typecheck are blocked in this fresh worktree
because generated/built `@hexclave/shared/dist/*` files are missing, and
repo instructions say not to run package builds from the agent.
- Full backend lint also reports an unrelated pre-existing error in
`apps/backend/scripts/run-bulldozer-studio.ts`.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Adds full Hexclave docs to `ask_hexclave` requests by fetching
https://docs.hexclave.com/llms-full.txt and appending them to the docs
agent system prompt. Includes a 5‑minute cache and 5s timeout, and skips
docs tools when `ask_hexclave` is used.
- **New Features**
- Added `getMcpSkillContextPrompt` to fetch and inject docs for
`ask_hexclave`; no‑op otherwise.
- Integrated in `route.ts` to append context before tool selection and
pass `mcpToolName` to `getTools`.
- Reliability: 5‑minute TTL cache, 5s timeout, and error handling; tests
cover success, no‑op, errors, timeouts, null/undefined, and cache hits.
- **Refactors**
- Switched source to `https://docs.hexclave.com/llms-full.txt`.
- Removed `docs-mintlify/llms-full.txt` and related generator code.
- Cache TTL now uses `performance.now()` for accurate expiry.
<sup>Written for commit e0dc388c64.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1605?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* AI queries now dynamically fetch and include documentation context
during operation, with in-memory caching to minimize network requests
and redundant fetches.
* **Tests**
* Added comprehensive test suite validating documentation fetching
behavior, error handling for network failures and timeouts, and caching
mechanisms to ensure reliability.
* **Chores**
* Removed auto-generated documentation artifact from the codebase.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mantra <mantra@stack-auth.com>
## What
Refresh the AI model selection matrix and fix a few issues in custom
dashboard generation.
### Models (`apps/backend/src/lib/ai/models.ts`)
- Replace deprecated/placeholder model IDs with current ones:
- `smart/slow` authenticated → `openai/gpt-5.5` (was
`x-ai/grok-build-0.1`)
- `smart/fast` → `google/gemini-3.5-flash`
- `smartest` unauthenticated tiers → `z-ai/glm-5.2` /
`google/gemini-3.5-flash` (was `deepseek/deepseek-v4-flash`)
- `dumb` unauthenticated tiers → `nvidia/nemotron-3-super-120b-a12b`
### Email template rewrite
- Forward `x-stack-*` / `x-hexclave-*` headers from the caller through
the template-source rewrite route so the inner AI call
(`/ai/query/generate`) is authenticated and resolves to the
**authenticated** model tier instead of falling back to the
unauthenticated one.
- Lower rewrite quality to `dumb` / `slow` (sufficient for this task,
cheaper/faster).
### Custom dashboard
- Speed up generation: `smart`/**fast** instead of `smart`/slow (both
`create-dashboard-preview.tsx` and `chat-adapters.ts`).
- Pin `@babel/standalone` to `7.29.7` in the sandbox host (avoid
surprise breakage from `latest`).
- Disable analytics in generated dashboards.
### Misc
- Bump MCP RPC timeout 15s → 45s (`apps/skills/src/mcp-wrapper.ts`).
## Testing
- `pnpm typecheck` ✅
- `pnpm lint` ✅
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Refreshes the model selection matrix, forwards auth headers so template
rewrites use authenticated tiers, and speeds up custom dashboard
generation with a more stable sandbox.
- **Refactors**
- Update model IDs: `openai/gpt-5.5`, `google/gemini-3.5-flash`,
`z-ai/glm-5.2`, `nvidia/nemotron-3-super-120b-a12b`.
- Use `openai/gpt-5.5` for authenticated fast routes.
- Forward `x-stack-*` / `x-hexclave-*` headers; build via Map to avoid
prototype-pollution; inner generate call uses the authenticated tier.
- Lower email template rewrite quality to `dumb`/`slow`.
- Switch dashboard generation to `smart`/`fast` in
`create-dashboard-preview.tsx` and `chat-adapters.ts`.
- Disable analytics in generated dashboards.
- Bump MCP RPC timeout from 15s to 45s.
- **Dependencies**
- Pin `@babel/standalone` to `7.29.7` in the sandbox host.
<sup>Written for commit 94354ae0f6.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1615?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Performance**
* Improved AI generation speed for dashboard creation and related chat
flows by using faster AI routing.
* Increased MCP JSON-RPC request timeout to better handle long-running
operations.
* **Technical**
* Template rewriting with AI now forwards authentication-related headers
to downstream AI calls for more consistent authorized behavior.
* Updated AI model routing/selection used by the proxy layer.
* **UI/Integration**
* Pinned the sandbox Babel CDN script to a specific version.
* Disabled analytics in the sandbox SDK configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
### Summary of Changes
Some stale data in bulldozer causes a price validation error which
causes a 500. We let it fail softly
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Fixes 500s in payments views by handling legacy product snapshots with
`prices: "include-by-default"` and other invalid price shapes. Stale
data now degrades gracefully, and we capture diagnostics instead of
crashing.
- **Bug Fixes**
- Normalize snapshot prices in `productToInlineProduct`: treat
`"include-by-default"` as `{}` and fall back to `{}` for any non-object;
capture errors for diagnostics.
- `productToInlineProduct` now accepts context (`productId`,
`customerType`, `customerId`); updated products and validate-code routes
to pass it.
- Added tests to verify price normalization and prevent response
validation failures.
<sup>Written for commit 9f34ad44a0.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1621?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved validation and error handling for product pricing data in
payment operations.
* Enhanced handling of malformed product snapshot data to ensure
stability.
* **Improvements**
* Strengthened product context consistency across payment endpoints and
purchase code validation flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
