Commit Graph

24 Commits

Author SHA1 Message Date
BilalG1
38ae913fc9
Rename STACK_* env vars to HEXCLAVE_* in env templates, with legacy dual-read (#1588)
Some checks failed
all-good: Did all the other checks pass? / all-good (push) Has been cancelled
Ensure Prisma migrations are in sync with the schema / check_prisma_migrations (22.x) (push) Has been cancelled
DB migration compat / Check if migrations changed (push) Has been cancelled
Docker Server Build and Push / Docker Build and Push Server (push) Has been cancelled
Docker Server Build and Run / docker (push) Has been cancelled
Runs E2E API Tests (Local Emulator) / E2E Tests (Local Emulator, Node ${{ matrix.node-version }}) (22.x) (push) Has been cancelled
Runs E2E API Tests / E2E Tests (Node ${{ matrix.node-version }}, Freestyle ${{ matrix.freestyle-mode }}) (mock, 22.x) (push) Has been cancelled
Runs E2E API Tests / E2E Tests (Node ${{ matrix.node-version }}, Freestyle ${{ matrix.freestyle-mode }}) (prod, 22.x) (push) Has been cancelled
Runs E2E API Tests with custom port prefix / build (22.x) (push) Has been cancelled
Runs E2E Fallback Tests / E2E Fallback Tests (Node ${{ matrix.node-version }}) (22.x) (push) Has been cancelled
Lint & build / lint_and_build (24) (push) Has been cancelled
TOC Generator / TOC Generator (push) Has been cancelled
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
DB migration compat / No migration changes (skipped) (push) Has been cancelled
## Summary

Completes the env-var side of the Hexclave rebrand: every
`STACK_*`-prefixed variable (including `NEXT_PUBLIC_STACK_*` and
`VITE_STACK_*`) is renamed to `HEXCLAVE_*` across all checked-in `.env`,
`.env.development`, and `.env.example` files (30 files, ~135 keys).
Legacy `STACK_*` names keep working everywhere via dual-read, so
**existing deployments, `.env.local` files, and self-hosted setups need
no immediate migration**.

## How legacy names keep working

- **Server code** already resolves `HEXCLAVE_*` first with `STACK_*`
fallback via `getEnvVariable`. Direct `process.env.STACK_X` readers fed
by the renamed files (prisma seed, e2e tests/helpers, internal-tool
scripts, examples, `prisma.config.ts`) now read `HEXCLAVE_X || STACK_X`.
- **Client code** (Next.js build-time inlining) uses literal dual-read
expressions; the dashboard's `_inlineEnvVars` already had them.
- **Docker/self-hosting**: `docker/server/entrypoint.sh` (shared by the
server and local-emulator images) gets a generic two-way
`HEXCLAVE_`↔`STACK_` env mirror — runs at startup and again before
sentinel replacement — replacing the previous URL-trio-only mirror.
Operators can use either prefix.

## The empty-placeholder trap (`||` vs `??`)

The checked-in templates define empty placeholders (`HEXCLAVE_X=#
comment` parses to `""` via dotenv). With `?? `-based fallbacks, that
empty string would silently shadow a real value under the legacy name —
including legacy vars set in Vercel/CI env at build time, since the
tracked `.env` is present during builds. All fallback chains therefore
treat empty-as-unset (`||`):

- `getEnvVariable` and `getProcessEnv` in `packages/shared`
- the dashboard/docs/example literal dual-reads
- the generated SDK env getters (via
`packages/template/scripts/generate-env.ts`; the generated
`src/generated/env.ts` files are gitignored and regenerate at build)

## Other notable changes

- Tests that override env now set the canonical `HEXCLAVE_*` name (it
wins over `STACK_*`): e2e `cross-domain-auth`, backend
`internal-feedback-emails` in-source test.
- e2e `helpers.ts` port-prefix expansion loop also matches the
`HEXCLAVE_` prefixes.
- `docker/local-emulator/generate-env-development.mjs` reads source keys
canonically (legacy fallback) and emits canonical keys; regenerated
output matches.
- `rotate-secrets.sh` falls back to
`HEXCLAVE_DATABASE_CONNECTION_STRING`.
- Docs code snippets (`docs/code-examples`) renamed outright to
canonical names, consistent with #1571.
- OAuth callback `console.warn` in `packages/template/src/lib/auth.ts`
now says Hexclave.

## Migration note for the team

Local `.env.local` files with legacy `STACK_*` overrides keep working
**unless** the override targets a var that `.env.development` now sets
to a real (non-empty) `HEXCLAVE_*` value — the canonical name wins over
file precedence. Rename those keys in your `.env.local` once.

## Verification

- `typecheck` + `lint` pass on every touched package (shared, backend,
dashboard, e2e, internal-tool, cli, docs, template). Pre-existing
failures on dev (`admin-app-impl.ts` typecheck, dashboard metrics-page
errors) are unchanged (identical error counts with/without this change).
- `getEnvVariable`/`getProcessEnv` fallback semantics smoke-tested
directly (empty-HEXCLAVE → legacy fallback, HEXCLAVE wins when set,
defaults intact).
- `internal-feedback-emails` in-source vitest passes; emulator env
generator `--check` passes; `bash -n` on touched shell scripts.
- Two independent review agents audited the diff for correctness bugs
and coverage gaps; all confirmed findings are fixed in the third commit.

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Renamed all `STACK_*` env vars (including
`NEXT_PUBLIC_STACK_*`/`VITE_STACK_*`) to `HEXCLAVE_*` across env
templates and code, with dual‑read that treats empty as unset, detects
conflicts, ignores post‑build sentinels, and falls back to legacy names.
All GitHub Actions now use `HEXCLAVE_*`; local‑emulator e2e is fixed by
setting `NEXT_PUBLIC_HEXCLAVE_IS_LOCAL_EMULATOR` in CI.

- **Refactors**
- Added conflict‑aware dual‑read helpers (prefer `HEXCLAVE_*`,
empty‑as‑unset, ignore post‑build sentinels, preserve empty passthrough)
and used them across `packages/shared` (resolver + tests),
`apps/dashboard` inline/public envs (with tests), `apps/backend` Prisma
config/seed and vitest (accept both prefixes), `packages/cli`
(API/Dashboard URLs, project ID, `HEXCLAVE_EMULATOR_HOME`; tests),
Docker (`entrypoint.sh` mirroring + `rotate-secrets.sh` DB URL),
docs/components (`docs/src/lib/env.ts`), and examples; hosted/Vite apps
now error if both spellings differ.
- Port‑prefix expansion includes `HEXCLAVE_*`; backend tests use a new
helper to resolve DB connection strings; Prisma prefers
`HEXCLAVE_DATABASE_CONNECTION_STRING` with legacy fallback.
- Generated SDK env getters use plain `HEXCLAVE_*` || `STACK_*` (no
conflict throw); dashboard inline resolver preserves empty/sentinel
passthrough to avoid build failures; docs/examples include dual‑read
utilities.
- Tests now stub canonical `HEXCLAVE_*` flags (e.g., plan limits, bot
challenge, OAuth tokens, hosted handler) to avoid shadowing/conflict
with committed defaults.

- **Migration**
  - No immediate action; legacy `STACK_*` names still work.
- If both names are set with different values, builds/scripts error. Set
only `HEXCLAVE_*` or make both equal.
- SDK consumers won’t see conflict throws; update env names to
`HEXCLAVE_*` over time.

<sup>Written for commit 7539fb9fbf.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1588?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Migrated environment variable names from the legacy `STACK_*` prefix
to the new `HEXCLAVE_*` prefix across backend, dashboard, tooling,
Docker, and examples.
* Updated environment/config resolution to prefer `HEXCLAVE_*`, treat
empty strings as unset, and detect conflicts when both `STACK_*` and
`HEXCLAVE_*` are set to different values.
* Updated local emulator, server startup, and env-generation workflows
to use the new names (with legacy fallback where applicable).
* **Documentation**
  * Updated docs and code examples to reference `HEXCLAVE_*` variables.
* **Tests**
* Refreshed unit and e2e coverage to validate dual-read behavior,
conflict detection, and empty-value handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-19 18:58:53 -07:00
Konsti Wohlwend
6a8b45e6c5
fix: CI failures — Node 22, test timeouts, QEMU transaction timeout (#1479) 2026-05-22 20:24:13 -07:00
Konsti Wohlwend
2e1a0d76e5
Pin all GitHub Actions to exact commit hashes (#1448) 2026-05-20 11:14:17 -07:00
Konstantin Wohlwend
c5436be6ac Skip setup tests on non-dev branches 2026-04-15 10:21:26 -07:00
Konstantin Wohlwend
c46767f4e8 Retry setup tests on non-staging branches to prevent flakeyness 2026-04-13 14:32:49 -07:00
Konsti Wohlwend
adb14f1635
[Refactor] Improve CI Run Times by Reducing Test Flakiness and Speeding up Test Suite (#1166) 2026-02-10 19:43:10 -08:00
aadesh18
2055d98dea
External db sync (#1036)
<img width="1920" height="969" alt="Screenshot 2026-02-04 at 9 47 16 AM"
src="https://github.com/user-attachments/assets/d7d0cd04-0051-4fc4-b857-e6f87ee97a59"
/>

**This PR revolves around the following components**
1. Sequencer - sequences the updates in the internal db
2. Poller - polls for the latest updates to sync with the external db
3. Outgoing Request Handler - essentially a trigger that can make http
requests based on a change in the internal db
4. Sync Engine - syncs with the latest changes from the internal db to
the external db

**What has been done**
- Added a global sequence id for ProjectUser, ContactChannel and
DeletedRow.
- Added the deletedRow table to keep track of the rows that were deleted
across ProjectUser and ContactChannel.
- Added the OutgoingRequest table to keep track of the outgoing requests
- Added function for the sequencer to call to sequence updates
- Added a sequencer that sequences all the changes in the internal db
every 50 ms
- Added a poller that polls for the latest changes in the internal db
every 50 ms, and adds to a queue
- Added a Vercel cron that calls sequencer and poller every minute
- Added a queue that fulfills the outgoing requests by making http calls
(for external db sync, it calls the sync engine endpoint)
- Added a sync engine that uses the defined sql mapping query in the
user's schema to pull in the changes for the user, and sync them with
the external db
- Added tests to test out each functionality


**How to review this PR:**
1. Review the migrations (sequence id, deletedRow, triggers, backlog
sync) (all files created under the migrations folder)
2. Review sequencer
3. Review poller
4. Review the changes in schema
5. Review sync-engine (the function, and it's helper file)
6. Review the schema changes, and query mappings
7. Review the tests (basic, advanced and race, along with the helper
file)
8. Review the changes made in Dockerfile to support local testing using
the postgres docker

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Introduces a cron-driven external DB sync pipeline with global
sequencing, internal poller and webhook sync engine, new DB
tables/functions, config schema/mappings, and comprehensive e2e tests.
> 
> - **Database (Prisma/Migrations)**:
> - Add global sequence (`global_seq_id`) and
`sequenceId`/`shouldUpdateSequenceId` to `ProjectUser`,
`ContactChannel`, `DeletedRow` with partial indexes.
> - Create `DeletedRow` (capture deletes) and `OutgoingRequest` (queue)
tables; add unique/indexes.
> - Add triggers/functions: `log_deleted_row`,
`reset_sequence_id_on_update`, `backfill_null_sequence_ids`,
`enqueue_tenant_sync`.
> - **Backend/API**:
> - New internal routes: `GET
/api/latest/internal/external-db-sync/sequencer`, `GET /poller`, `POST
/sync-engine` (Upstash-verified) for sync orchestration.
> - Add cron wiring: `vercel.json` schedules and local
`scripts/run-cron-jobs.ts`; start in dev via `dev` script.
> - Tweak route handler (remove noisy logging) without behavior change.
> - **Sync Engine**:
> - Implement `src/lib/external-db-sync.ts` to read tenant mappings and
upsert to external Postgres (schema bootstrap, param checks,
sequencing).
> - Add default mappings `DEFAULT_DB_SYNC_MAPPINGS` and config schema
`dbSync.externalDatabases` in shared config.
> - **Testing/Infra**:
> - Add extensive e2e tests (basics, advanced, race conditions) for
sequencing, idempotency, deletes, pagination, multi-mapping, and
permissions.
> - Docker compose: add `external-db-test` Postgres for tests; e2e deps
for `pg` types.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
3f2a8efcfb. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* External PostgreSQL sync: automatic, batched replication with
mappings, resume/idempotency, and on-demand enqueueing.

* **Admin UI**
* Real-time External DB Sync dashboard and status API showing
per-mapping backlog, sequencer/poller/sync-engine telemetry, and fusebox
controls.

* **Tests**
* Large e2e suite: basic, advanced, race, high-volume tests and test
utilities for external DB sync.

* **Chores**
* DB migrations, CI/workflow updates, background cron runner and
local/dev test support.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>
Co-authored-by: Bilal Godil <bg2002@gmail.com>
2026-02-05 12:04:31 -08:00
BilalG1
7b5cf4f042
fix analytics queries (#1141)
<!--

Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md

-->


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Improvements**
* Added automatic retry with stricter result validation and clearer
error handling for query profiling to improve reliability.

* **Chores**
* CI workflows updated to use larger runner instances (upgraded runner
size across relevant jobs).

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-29 10:21:58 -08:00
Konstantin Wohlwend
5421e6dd38 Upgrade pnpm in GH workflows 2025-11-24 12:02:09 -08:00
Konstantin Wohlwend
8580a1ea14 Upgrade GitHub workflow scripts 2025-11-24 11:21:43 -08:00
Konstantin Wohlwend
0e05662e4e Increase test workers 2025-03-19 15:13:24 -07:00
Konstantin Wohlwend
2605850f95 Increase Vitest test timeout 2025-03-19 15:12:05 -07:00
Konstantin Wohlwend
217e6da8b6 Verbose test outputs 2025-03-19 14:07:21 -07:00
Konstantin Wohlwend
656bf4c44b chore: update package versions 2025-03-19 12:54:10 -07:00
Konstantin Wohlwend
130c8e648e Fix CI 2025-03-19 09:32:10 -07:00
Konstantin Wohlwend
79668811db Disable file parallelism for dev tests 2025-03-18 13:13:52 -07:00
Konstantin Wohlwend
8cc3393110 GH Actions push trigger only on main or dev branches 2025-03-12 10:52:35 -07:00
Konstantin Wohlwend
bd687e8c43 Re-add pull request trigger to GH Actions 2025-03-12 10:43:36 -07:00
Konstantin Wohlwend
7c3c4863ab Fix GitHub Actions concurrency 2025-03-10 14:09:09 -07:00
Konstantin Wohlwend
a2dbecb606 Cancel in-progress GitHub Actions on feature branches 2025-03-10 12:14:58 -07:00
Konsti Wohlwend
b41681d1e4
Reduce test flakeyness (#517)
Some checks failed
All good? / all-good (push) Has been cancelled
Update pull request branches / Update pull request branches (push) Has been cancelled
Ensure Prisma migrations are in sync with the schema / check_prisma_migrations (22.x) (push) Has been cancelled
Docker Build and Push / Docker Build and Push Server (push) Has been cancelled
Docker Test / docker (push) Has been cancelled
Runs E2E API Tests / build (20.x) (push) Has been cancelled
Runs E2E API Tests / build (22.x) (push) Has been cancelled
Runs E2E API Tests / build (latest) (push) Has been cancelled
Lint & build / lint_and_build (20.x) (push) Has been cancelled
Lint & build / lint_and_build (22.x) (push) Has been cancelled
Mirror main branch to main-mirror-for-wdb / lint_and_build (push) Has been cancelled
Publish Docs / run (push) Has been cancelled
Dev Environment Test / test (push) Has been cancelled
Run setup tests / test (push) Has been cancelled
TOC Generator / TOC Generator (push) Has been cancelled
2025-03-07 13:34:50 -08:00
Zai Shi
427ab34adb
Codegen + macros for SDK docs (#456) 2025-02-27 23:12:30 +01:00
Konstantin Wohlwend
02d1680d2e Run GitHub Actions on push instead of PR 2025-02-11 09:59:18 -08:00
CactusBlue
e27173d27e
Automated testing for setup instructions in README.md (#399)
* add setup tests

* continue, even on error

* add build:packages

* add automated branch for setup testing

* add delay

* combine commands

* rollback

* update README

* use action to run in background

* try running in foreground for testing

* set shell

* check if running codegen before works

* fix quetzal file missing

* swap the order

* use background action

* fix

* try this instead

* modify readme

* add sleep

* raise the timeout

* run on faster machine

* use jarvus instead

* Update packages/stack/scripts/merge-quetzal-translations.ts

Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>

* Update README.md

Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>

* fix run on all branches

* Update .github/workflows/setup-tests.yaml

Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>

---------

Co-authored-by: root <TheCactusBlue>
Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>
2025-01-27 16:48:38 -08:00