Docker is difficult to run on macOS CI runners (colima VZ and QEMU
backends both crash). Split into two stages:
1. docker-build (Linux): builds arm64 Docker image, exports tarball
2. qemu-snapshot (macOS): provisions QEMU VM under HVF, captures snapshot
Add SKIP_DOCKER_BUILD=1 to build-image.sh to reuse a pre-built bundle.
Cross-arch arm64-on-amd64 docker buildx runs the rolldown-backed tsdown
build under qemu-user, whose futex emulation occasionally deadlocks the
worker threads. Wrap the call in a bounded timeout + 3-attempt retry so
a hang fails the layer in <11min and recovers on the next try.
Snapshot-resume mode dropped virtfs (QEMU disallows migration with 9p
mounted), leaving the container's /host bind mount empty so the
/local-emulator/project route returned 400. Hot-plug virtio-9p over a
pre-attached PCIe root port after resume, mount in guest via QGA, and
make /host a shared mount point with rshared docker propagation so the
new mount reaches the running container without restart.
run-emulator.sh ensure_runtime_config_iso now falls back to in-script
ISO generation when invoked outside the CLI (fixes pnpm emulator:start).
Also propagate fresh emulator credentials to VITE_/EXPO_PUBLIC_ env var
prefixes in the CLI run command.
- run-emulator.sh: drop wait_for_condition poll interval from 1s to 0.2s
- emulator.ts: replace existsSync+readFileSync TOCTOU in readInternalPck
with try/ENOENT; tighten initial backoff to 50ms; drop redundant
mkdirSync in startEmulator; surface stop-failure on stderr instead of
swallowing silently
- iso.ts: inline trivial buildRootDirRecordInVD wrapper
- stop_vm no longer deletes runtime-config.iso; the CLI owns its
lifecycle and the snapshot → cold-boot fallback needs it preserved
(cmd_reset still wipes RUN_DIR for a full reset). Also sweeps qga.sock.
- Write internal-pck to \$VM_DIR on the host in snapshot mode. Cold boot
publishes this via virtfs/9p; snapshot mode drops virtfs, so
--config-file flows would otherwise hang. Handles both the rotation
path (fresh PCK) and EMULATOR_NO_ROTATION (placeholder PCK).
- Pin RAM in snapshot mode to the build-time 4096 (overridable via
EMULATOR_SNAPSHOT_RAM). Migration replay requires an identical -m
value, same constraint as CPU count.
- Fail amd64 build when .savevm.zst is missing rather than shipping a
cold-boot-only release silently. arm64 stays best-effort for now
because it runs under TCG and can't be verified end-to-end.
- Install Node/pnpm on both arches. arm64 also runs
generate-env-development.mjs, which otherwise relied on the runner
image's preinstalled Node.
The docker/server image runs as the unprivileged `node` user, which
cannot write to /var/run. With `set -e` at the top of the script, the
failed `touch` aborted execution after sentinel replacement but before
the backend/dashboard were started — the Check server health CI step
then saw connection refused on ports 8101/8102.
Move the marker into $WORK_DIR (which is already created and owned by
the running user). The emulator snapshot-resume path still benefits: the
marker persists across supervisorctl restarts because $WORK_DIR lives
on the container filesystem.
Snapshot resume drops from ~14s to ~5-7s with rotation, ~2.5s without.
Build uses QEMU's mapped-ram + multifd migration capability so the RAM
state is written at page-aligned offsets in a sparse file. Runtime
decompresses the shipped .savevm.zst once to a local .raw cache and
reloads via -incoming file: + migrate-incoming on subsequent starts,
avoiding the per-start zstd decode.
Adds EMULATOR_NO_ROTATION=1 for tests/CI that don't mind the placeholder
secrets; saves the full ~3s rotation window.
Misc runtime cleanups: tighter QMP/QGA poll intervals (1s → 0.2s),
shorter socat keep-alive windows, 1s settle before the post-rotation
health-check to avoid racing old Node processes, fallback path preserves
the CLI-generated runtime-config.iso instead of blowing away VM_DIR.
Build-time qmp_session keeps stdin open briefly after the caller's
commands so migrate-set-capabilities is actually processed before
socat closes — without this, mapped-ram was silently a no-op.
CI workflow publishes .savevm.zst alongside the .qcow2 (optional asset;
CLI falls back to cold boot when missing). Test + verify steps go
through the CLI now that ISO generation is owned by packages/stack-cli.
Ships a compressed RAM/device snapshot (stack-emulator-<arch>.savevm.zst)
alongside the qcow2. `emulator start` resumes from it and rotates the
per-install secrets in place, taking cold-boot from 30-120s to ~6-7s.
Build phase adds a STACKCFG runtime ISO so stack.service can boot during
image creation, starts qemu-guest-agent so its virtio-serial port stays
open in the snapshot, then stop+migrate file:+quit via QMP.
Runtime sends fresh secrets through QGA guest-exec input-data, which pipes
them to trigger-fast-rotate and rotate-secrets inside the container:
targeted sed on the placeholder PCK in built JS, UPDATE on the internal
ApiKeySet, supervisorctl restart stack-app + cron-jobs. Placeholder hex
values are baked in instead of random keys under STACK_EMULATOR_BUILD_SNAPSHOT=1
so no real secret ships in the snapshot.
Device topology and SMP must match at capture and resume; runtime adds
phantom seed/bundle drives and pins SMP=4. Cold-boot fallback kicks in
automatically when the snapshot is missing, corrupt, or incompatible.
supervisord.conf now uses stopasgroup/killasgroup for stack-app and
cron-jobs so supervisor restart actually kills the Node children (they
were keeping their port bindings and breaking rotation).
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added Stripe, OAuth, and Freestyle mock services to the local emulator
* Introduced `emulator run` CLI command to execute applications with
emulator credentials automatically injected
* Enhanced credential management for local development
* **Improvements**
* Improved ARM64 QEMU emulation with cross-architecture support
* Better error detection and logging during emulator provisioning
* Added example middleware configuration with authentication support
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
… V8 --jitless
2.6 GB to 1.3 GB final image
Flip arm64 matrix back to ubicloud-standard-8 so both arches share one
runner fleet. Cross-arch TCG on an amd64 host previously SIGTRAP'd in
migrations because V8's JIT emitted arm64 instructions that QEMU's
cross-arch translator mis-handled; pair the existing -cpu cortex-a72
fallback with NODE_OPTIONS=--jitless on the migration docker exec to
force V8 to stay on the interpreter. Does not affect amd64 migrations
(KVM, no TCG).
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Optimized emulator images with binary stripping, compression, and
preservation of standalone runtime dependencies.
* Improved multi-architecture build matrix, added optional KVM
detection/fallback, and gated certain emulator runtime steps for arm64.
* Enhanced build scripts to generate and include env files and persist
richer logs and artifacts.
* **New Features**
* Centralized provision entrypoint to streamline install → migrations →
slimming sequence.
* **Tests**
* Added a fast QEMU serial boot test for architecture validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Added support for `@opentelemetry/sdk-node` in the backend.
- Updated various dependencies including AWS SDK and OpenTelemetry
packages.
- Implemented graceful shutdown handling for non-Vercel runtimes in
`prisma-client.tsx`.
- Enhanced AWS credentials retrieval to support GCP Workload Identity
Federation.
- Introduced a Dockerfile for Cloud Run deployment, optimizing the
backend build process.
- Updated `.gitignore` to include Terraform runtime files and secrets.
This commit improves the backend's observability and deployment
flexibility, particularly for Cloud Run environments.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* OpenTelemetry observability with dynamic provider selection per
deployment.
* Cloud Run trusted-proxy support for accurate client IP handling.
* Graceful shutdown that waits for in-flight background work.
* New background-task handling to improve async webhook/email delivery
reliability.
* AWS credential providers added (Vercel OIDC & GCP Workload Identity
Federation).
* Dockerized backend image for Cloud Run / self-host deployments.
* **Chores**
* Updated dependencies for OpenTelemetry and AWS SDK support.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
s3mock 5.0.0 was released today and latest now points to it. The major
version bump breaks initialBuckets env var handling, causing
NoSuchBucket errors in CI.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Pinned service dependency to a specific version for improved
deployment consistency and stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
commit 5d43722575b826a8ed8dbb6b828f48eae4bca02c
Author: mantrakp04 <mantrakp@gmail.com>
Date: Wed Mar 18 12:27:01 2026 -0700
Add QEMU emulator snapshot functionality and reset command
- Introduced a new `emulator-qemu:reset` command in package.json to
clear snapshots and force a fresh boot of the emulator.
- Enhanced the `run-emulator.sh` script to support saving and restoring
snapshots, significantly reducing restart time from ~62s to ~4s.
- Implemented logic to check for existing snapshots and restore them
during startup, improving the emulator's efficiency.
- Updated documentation in CLAUDE-KNOWLEDGE.md to explain the new
snapshot restore process and its benefits.
These changes enhance the QEMU emulator's performance and usability for
developers, providing a more efficient workflow during development.
commit 3877445bdd83cb8690da18c8520bf260d2795172
Author: mantrakp04 <mantrakp@gmail.com>
Date: Wed Mar 18 11:55:18 2026 -0700
Enhance QEMU emulator performance and configuration management
- Added optimizations to the QEMU emulator's app container startup
process, reducing startup time from ~92s to ~62s by using qcow2 backing
files and setting the working directory to /app.
- Updated the build-image.sh script to conditionally wait for background
processes, improving robustness.
- Modified the run-emulator.sh script to create the disk image using
qcow2 format instead of copying, enhancing efficiency.
- Adjusted the cloud-init user-data to set STACK_RUNTIME_WORK_DIR to
/app, streamlining file operations during container initialization.
- Improved the entrypoint script to avoid unnecessary file copying when
the working directory is set to /app.
These changes significantly enhance the performance and usability of the
QEMU emulator for developers.
commit e0b86d3f1d5c08e46d0d343bc632e2a8c5777845
Author: mantrakp04 <mantrakp@gmail.com>
Date: Wed Mar 18 11:07:55 2026 -0700
Refactor local emulator configuration management and enhance Docker
setup
- Removed redundant comments and improved code clarity in the local
emulator's route handling.
- Streamlined the Dockerfile and docker-compose.yaml for better
readability and maintenance.
- Updated entrypoint and initialization scripts to enhance service
startup processes.
- Introduced a new common script for QEMU emulator to centralize
architecture detection and firmware handling.
- Enhanced error handling in the host file bridge for improved
robustness.
- Removed obsolete country code utilities to clean up the codebase.
These changes significantly improve the local emulator's configuration
management and overall setup experience for developers.
commit 4fb0f93c6cc4f749a14acf0228c261e180875609
Author: mantrakp04 <mantrakp@gmail.com>
Date: Wed Mar 18 10:24:53 2026 -0700
Implement local emulator file bridge for enhanced configuration
management
- Introduced a new host file bridge to facilitate reading and writing
configuration files between the local emulator and the host system.
- Refactored the local-emulator module to utilize the file bridge for
file operations, improving error handling and response validation.
- Added tests to ensure the file bridge functionality works as expected,
including handling of non-existent files and writing configurations.
- Updated the run-emulator script to start the file bridge
automatically, ensuring seamless integration during emulator startup.
- Enhanced documentation to reflect the new file bridge capabilities and
usage instructions.
These changes significantly improve the local emulator's ability to
manage configuration files, enhancing the development experience.
commit 3d18a7ce5bbf00a62a40a3f48f27856e79ecc62f
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 22:36:46 2026 -0700
Refactor QEMU local emulator setup and enhance app bundle handling
- Introduced a new script for packaging Docker images into a compressed
app bundle, improving the emulator's deployment process.
- Updated build-image.sh to create a runtime configuration ISO, ensuring
better management of environment settings.
- Enhanced cloud-init user-data scripts for both dev-server and deps
guests, streamlining service setup and configuration.
- Improved the run-emulator.sh script to facilitate better handling of
runtime configurations and dependencies.
- Adjusted the .gitignore to include .DS_Store and removed obsolete
entries, cleaning up the repository.
These changes significantly enhance the local emulator's functionality
and reliability for developers.
commit 8a35fb1ce79898d73e2259e256c11b6fd9b0a584
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 21:52:24 2026 -0700
Enhance local emulator functionality and configuration
- Updated package.json to improve the start-emulator command, providing
clearer dashboard and backend URLs.
- Added a new wait-until-emulator-is-ready command to ensure the
emulator is fully operational before proceeding.
- Refactored the local-emulator project route to streamline file
existence checks and default config creation.
- Enhanced user guidance in the dashboard for local Stack config file
handling.
- Updated tests to reflect changes in config file handling, ensuring
non-existent files are created with default settings.
- Improved Docker configurations for the local emulator, including new
environment variables and service dependencies.
These changes significantly enhance the local development experience and
emulator reliability.
commit 3910ed4bc40bbb37340c1c316c24c2826ba372bd
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 19:59:36 2026 -0700
Remove unused stash-0.patch file to clean up the repository.
commit 74146d974458037a7a9590120a524629a1a6a162
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 19:58:46 2026 -0700
Enhance QEMU local emulator with app bundle support and runtime
configuration
- Introduced a new script to package the backend and dashboard assets
into a standalone app bundle for the QEMU emulator.
- Updated the build-image.sh script to create an ISO containing the app
bundle, ensuring the guest image includes the full runtime.
- Modified cloud-init user-data to handle the new app bundle and runtime
configuration, improving the setup process for local development.
- Enhanced the run-emulator.sh script to prepare and mount the runtime
configuration ISO, facilitating better environment management for the
emulator.
- Updated the user-data to include necessary environment variables for
the stack application, ensuring seamless integration during startup.
These changes significantly improve the local emulator's functionality
and ease of use for developers.
commit 9e865a1cf524398bc58f00e0836278775c4ae936
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 16:50:45 2026 -0700
Enhance local emulator setup with new services and configurations
- Added Docker support for a local emulator, integrating PostgreSQL,
Redis, Inbucket, Svix, ClickHouse, MinIO, and QStash.
- Introduced new scripts for managing the emulator lifecycle, including
build and run commands.
- Implemented cloud-init provisioning for automatic service setup on
first boot.
- Updated package.json with new commands for emulator management and
added dotenv-cli for environment variable management.
- Added tests for OAuth authorization flow to return JSON responses.
- Included configuration files for ClickHouse and user management.
This commit significantly improves the local development experience by
providing a comprehensive emulator environment.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced a local QEMU-based emulator for development with bundled
services (PostgreSQL, Redis, ClickHouse, MinIO, Inbucket, Svix, QStash).
* Added CLI commands to manage the emulator (start, stop, reset, status,
pull images).
* Added emulator status dashboard to monitor service health.
* Introduced new configuration system via `stack.config.ts`.
* **Tests**
* Added configuration read/write tests for the emulator.
* Added emulator CLI validation tests.
* **Documentation**
* Added emulator setup and usage guide.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Enhances sign-up process with Turnstile integration for fraud
protection. Builds on top of fraud-protection-temp-emails.
Made with [Cursor](https://cursor.com)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Cloudflare Turnstile bot-protection across signup/sign-in flows
(including SDK JSON mode).
* Email deliverability checks via Emailable.
* Sign-up risk scoring with persisted risk metrics and country code
tracking.
* UI: country-code selector, risk-score editing in user details, users
list refresh button, and Turnstile signup demo pages.
* **Bug Fixes**
* Use actual sign-up timestamp for reporting/metrics.
* **Documentation**
* Expanded knowledge base on Turnstile, risk scoring, and env
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
Co-authored-by: BilalG1 <bg2002@gmail.com>
Co-authored-by: Armaan Jain <84474476+Developing-Gamer@users.noreply.github.com>
Co-authored-by: nams1570 <amanganapathy@gmail.com>
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Provision local-emulator projects from a local config file and return
emulator credentials via a new internal endpoint.
* Dashboard: "Open config file" flow to open local projects and refresh
owned projects.
* **Changes**
* Branch config can prefer/read/write local files for emulator projects.
* Environment config updates/resets are blocked for local-emulator
projects.
* Dashboard UI shows read-only notices and disables project creation in
emulator mode.
* Added DB mapping and a standard env flag to identify local-emulator
projects.
* **Tests**
* New E2E tests covering provisioning and config restrictions.
* **Chores**
* Removed legacy emulator docs and compose; added CI workflow for
local-emulator E2E runs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
https://www.loom.com/share/3b7c9288149e4f878693281778c9d7e0
## Todos (future PRs)
- Fix pre-login recording
- Better session search (filters, cmd-k, etc)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Analytics → Replays: session recording & multi-tab replay with
timeline, speed, seek, and playback settings; dashboard UI for listing
and viewing replays.
* **Admin APIs**
* Admin endpoints to list recordings, list chunks, fetch chunk events,
and retrieve all events (paginated).
* **Client**
* Client-side rrweb recording with batching, deduplication, upload API
and a send-batch client method.
* **Configuration**
* New STACK_S3_PRIVATE_BUCKET for private session storage.
* **Tests**
* Extensive unit and end-to-end tests for replay logic, streams,
playback, and APIs.
* **Chores**
* Removed an E2E API test GitHub Actions workflow.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<img width="1920" height="969" alt="Screenshot 2026-02-04 at 9 47 16 AM"
src="https://github.com/user-attachments/assets/d7d0cd04-0051-4fc4-b857-e6f87ee97a59"
/>
**This PR revolves around the following components**
1. Sequencer - sequences the updates in the internal db
2. Poller - polls for the latest updates to sync with the external db
3. Outgoing Request Handler - essentially a trigger that can make http
requests based on a change in the internal db
4. Sync Engine - syncs with the latest changes from the internal db to
the external db
**What has been done**
- Added a global sequence id for ProjectUser, ContactChannel and
DeletedRow.
- Added the deletedRow table to keep track of the rows that were deleted
across ProjectUser and ContactChannel.
- Added the OutgoingRequest table to keep track of the outgoing requests
- Added function for the sequencer to call to sequence updates
- Added a sequencer that sequences all the changes in the internal db
every 50 ms
- Added a poller that polls for the latest changes in the internal db
every 50 ms, and adds to a queue
- Added a Vercel cron that calls sequencer and poller every minute
- Added a queue that fulfills the outgoing requests by making http calls
(for external db sync, it calls the sync engine endpoint)
- Added a sync engine that uses the defined sql mapping query in the
user's schema to pull in the changes for the user, and sync them with
the external db
- Added tests to test out each functionality
**How to review this PR:**
1. Review the migrations (sequence id, deletedRow, triggers, backlog
sync) (all files created under the migrations folder)
2. Review sequencer
3. Review poller
4. Review the changes in schema
5. Review sync-engine (the function, and it's helper file)
6. Review the schema changes, and query mappings
7. Review the tests (basic, advanced and race, along with the helper
file)
8. Review the changes made in Dockerfile to support local testing using
the postgres docker
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces a cron-driven external DB sync pipeline with global
sequencing, internal poller and webhook sync engine, new DB
tables/functions, config schema/mappings, and comprehensive e2e tests.
>
> - **Database (Prisma/Migrations)**:
> - Add global sequence (`global_seq_id`) and
`sequenceId`/`shouldUpdateSequenceId` to `ProjectUser`,
`ContactChannel`, `DeletedRow` with partial indexes.
> - Create `DeletedRow` (capture deletes) and `OutgoingRequest` (queue)
tables; add unique/indexes.
> - Add triggers/functions: `log_deleted_row`,
`reset_sequence_id_on_update`, `backfill_null_sequence_ids`,
`enqueue_tenant_sync`.
> - **Backend/API**:
> - New internal routes: `GET
/api/latest/internal/external-db-sync/sequencer`, `GET /poller`, `POST
/sync-engine` (Upstash-verified) for sync orchestration.
> - Add cron wiring: `vercel.json` schedules and local
`scripts/run-cron-jobs.ts`; start in dev via `dev` script.
> - Tweak route handler (remove noisy logging) without behavior change.
> - **Sync Engine**:
> - Implement `src/lib/external-db-sync.ts` to read tenant mappings and
upsert to external Postgres (schema bootstrap, param checks,
sequencing).
> - Add default mappings `DEFAULT_DB_SYNC_MAPPINGS` and config schema
`dbSync.externalDatabases` in shared config.
> - **Testing/Infra**:
> - Add extensive e2e tests (basics, advanced, race conditions) for
sequencing, idempotency, deletes, pagination, multi-mapping, and
permissions.
> - Docker compose: add `external-db-test` Postgres for tests; e2e deps
for `pg` types.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
3f2a8efcfb. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* External PostgreSQL sync: automatic, batched replication with
mappings, resume/idempotency, and on-demand enqueueing.
* **Admin UI**
* Real-time External DB Sync dashboard and status API showing
per-mapping backlog, sequencer/poller/sync-engine telemetry, and fusebox
controls.
* **Tests**
* Large e2e suite: basic, advanced, race, high-volume tests and test
utilities for external DB sync.
* **Chores**
* DB migrations, CI/workflow updates, background cron runner and
local/dev test support.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>
Co-authored-by: Bilal Godil <bg2002@gmail.com>
<!-- CURSOR_SUMMARY -->
> [!NOTE]
> **High Risk**
> Touches core sign-up/auth flows and user restriction semantics
(including new DB constraints) and introduces dynamic rule
evaluation/logging; misconfiguration or CEL/parser bugs could block
sign-ups or incorrectly restrict users.
>
> **Overview**
> Introduces **CEL-based sign-up rules** (config-driven) that are
evaluated during password/OTP/OAuth sign-ups and anonymous upgrades;
matching rules can reject sign-ups or mark users as admin-restricted,
and triggers are logged for analytics.
>
> Extends `ProjectUser` with `restrictedByAdmin` plus public/private
restriction details, updates restriction computation/filtering, and
exposes these fields via user CRUD (including validation + DB constraint
enforcing consistency when unrestricted).
>
> Adds a new dashboard **Sign-up Rules** page with a visual condition
builder (CEL <-> visual tree), drag-reorder by priority, per-rule 48h
sparkline analytics via a new hidden internal endpoint, and adds
user-page UI to view/edit manual restrictions. Also refactors ClickHouse
client initialization to require env vars (removing
`isClickhouseConfigured` checks) and adjusts CI container startup wait
time.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2141e689e8c1b72303b805e9234f996010d0880. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Sign-up Rules: visual rule builder, in-project CRUD with drag-reorder,
per-rule analytics, backend evaluation, and admin UI.
* Admin user restrictions: dashboard controls, banners/status,
public/private admin details surfaced in user views.
* **APIs & Schema**
* Config and user schemas extended; new SignUpRejected error and sign-up
rule types added.
* **Tests**
* Extensive unit and E2E coverage for rules, parser, evaluator,
analytics, and restricted-user flows.
* **Docs**
* Editorial guidance added to AGENTS.md.
* **Chores**
* DB statement timeout, updated clean script, minor dependency
additions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
### Context
We noticed some errors pop up on sentry related to email rendering.
These errors seem to have been triggered by the same issue, and could be
categorized as follows:
1. Sanity test mismatch, even when the errors from freestyle and vercel
sandbox were broadly similar. This occurred due to stack traces
differing in different execution environments.
2. Rendering errors from freestyle and vercel sandbox caused by the
theme not being imported/ empty theme component.
Upon investigation, this occurred because hitting save on the email
themes page with an invalid theme (ex: deleting the `export` keyword, or
renaming the `EmailTheme` component) still triggers `bundleAndExecute`
with the invalid themes. This will obviously fail and cause the errors
to be logged, however there is no cause for concern here because the
error is returned and the save is denied because an error is returned.
It's more of a matter of noisy error logs and too strict sanity test
comparisons.
Beyond that, `js-execution` is a little opaque and hard to understand,
and this can mask errors in logic.
We also noticed a new issue: manually throwing an error in the email
theme code editor, and then trying to save was actually successful. This
was because the version of `react-email/components` we were using had
faulty error handling, and fell back to client side rendering, masking
the error. This wasn't caught by our `try-catch` safeguards because it
was a render time issue that was masked. More specifically, this was
what `react-email` was doing: `Switched to client rendering because the
server rendering errored`.
### Summary of Changes
We loosen the sanity test comparison between engine execution results in
case of errors. We then refactor the `js-execution` and
`email-rendering` files to read better, and to only `captureError` when
a service is down, but not for runtime errors in the user submitted
code.
To deal with the other bug, we bumped `react-email/components` to the
latest version. However, doing so exposed a gap between real `freestyle`
and our `freestyle-mock`: with the mock, the errors that were now raised
were treated as uncaught exceptions, crashing the mock server.
Consequently, we switched to using `node` over `bun`.
We also expanded test coverage to account for different error paths.
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
Conditionally generate secrets. This stops docker image from generating
new secrets upon every restart.
Originally reported in #578.
This fix aims to resolve this issue.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Secret values can now be externally injected during startup without
being overwritten. Pre-configured secrets are preserved instead of being
regenerated.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
We update the sdk dependencies (the ones present in
`package-template.json`) to the latest versions. Since several packages
have major version bumps, this results in a variety of breaking changes
that have been handled here. Incidentally, when possible, we update
similar dependencies across the codebase.
We decide to defer the tailwind update to another PR owing to its scale.
The rest of the updates and changes have been catalogued below:
1.
[Bumping](https://github.com/panva/oauth4webapi/blob/v3.x/CHANGELOG.md)
`oauth4webapi` to 3.8.3: this was a major version changed. While there
were no compatibility issues in the sdk, there were several breaking
changes in `stack-shared`. Namely:
a. The removal of `isOauth2Error`. We used this to check if the results
of our `oauth4webapi` api invocations had issues. The functions were
changed to explicitly throw either `ResponseBodyErrors` or
`AuthorizationResponseErrors`, so the code was reworked to account for
that with no loss in error handling.
b. Dropping of support for http broadly: `oauth4webapi` now only accepts
https. This is desired, but I add a carve out for our test environments
only.
c. `refreshTokenGrantRequest` and `authorizationCodeGrantRequest` now
require `clientAuthentication` to be passed explicitly to them.
d. Changes in how we handle our `MultiFactorAuthenticationRequired`
error: This is an error that we created and is passed to the
`oauth4webapi` API if there are MFA issues. Since the
`processAuthorizationCodeResponse` now explicitly throws a
`ResponseBodyError`, we access the error cause from the body of the
error instead.
2. [Bumping](https://github.com/Qix-/color/releases) `color` to 5.0.4:
this was a major version bump. Simple type checking change, I checked
the API for the correct interface.
3.
[Bumping](https://github.com/MasterKale/SimpleWebAuthn/blob/master/CHANGELOG.md)
`simplewebauthn` to 13.2.2: two major version bumps, but no
incompatibilities surprisingly
4. [Bumping](https://github.com/jshttp/cookie/releases) `cookie` to
1.1.1: this was a major version bump.
a. Changing `parse` to `parseCookie`. In the most recent version,
`parse` is still maintained as an alias for `parseCookie` for backwards
compatibility, but I thought it would be best to change it over now. No
change in functionality.
b. Typing is now strongly enforced. A cookie can be `string |
undefined`, and the `Cookies` are now `Record<string, string |
undefined>`. We already have code to handle if a cookie is returned as
undefined/ null, so the changes here were more to ensure type
compatibility rather than big changes in functionality.
5. [Bumping ](https://github.com/isaacs/rimraf#readme)`rimraf` to 6.1.2:
No breaking changes, mostly just bug fixes.
6. [Bumping](https://github.com/panva/jose/releases?page=1) `jose` to
6.1.3: This is another major version bump. We update it across the
codebase to ensure compatibility. We use this for importing and
processing jwk tokens. There are a few big changes in the version bump,
but the only one that applies to us is that `importJwk` now yields a
`CryptoKey` instead of a `KeyObject` in Node.js. However, this doesn't
appear to break our code. We use `importJwk` in
`stack-auth/packages/stack-shared/src/utils/jwt.tsx`.
7. [Bumping](https://github.com/react-hook-form/resolvers/releases)
`hookform/resolvers` to 5.2.2 (two major version jumps), and
consequently bumping `react-hook-form` to 7.70.0: We already use the
patterns that `hookform/resolvers`' latest versions seem to be
enforcing. The only other breaking change is that it requires version
7.55.0+ of `react-hook-form`. Though we should pay attention to any
interactions with zod and `hookform/resolvers`, some people have
reported compatibility issues if they aren't using the latest compatible
versions of both.
8. [Bumping](https://github.com/jquense/yup/blob/master/CHANGELOG.md)
`yup` to 1.7.1: this was a minor version change, but we had
incompatibility issues with this change. Versions 1.4.1 and 1.7.1 cannot
exist in the same codebase due to incompatibility, so we bumped it up
across the codebase, including in peer dependencies.
9. Some minor version changes for some packages, but these were mostly
bug fixes.
10. **Edited to add**: Bumping freestyle to 0.1.6, and reworking the
freestyle mock server. In 0.1.6, freestyle changed their API in two
ways:
a. We're now supposed to hit their `execute/v2/...` endpoint and
b. They've flattened the `config` argument to `serverless.runs.create`.
These changes are minor, but are important. As part of a general suite
of dependency bumps, this was judged to fit here.
We have linked the changelogs for the packages on each line.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added environment variable handling and logging interception for
improved script execution visibility.
* **Bug Fixes**
* Enhanced error handling with structured error responses and logs.
* **Chores**
* Updated dependencies: arktype and react-dom.
* Improved temporary work directory management and cleanup process.
* Optimized script execution model for better performance.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!--
ONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- RECURSEML_SUMMARY:START -->
## High-level PR Summary
This PR changes the default development ports for several background
services to avoid conflicts. PostgreSQL moves from port `5432` to
`8128`, Inbucket SMTP from `2500` to `8129`, Inbucket POP3 from `1100`
to `8130`, and the OpenTelemetry collector from `4318` to `8131`. All
references across configuration files, Docker Compose setups,
environment files, CI/CD workflows, test files, and documentation have
been updated to reflect these new port assignments. A knowledge base
document has been added to document the new port mappings.
⏱️ Estimated Review Time: 15-30 minutes
<details>
<summary>💡 Review Order Suggestion</summary>
| Order | File Path |
| --- | --- |
| 1 | `claude/CLAUDE-KNOWLEDGE.md` |
| 2 | `apps/dev-launchpad/public/index.html` |
| 3 | `docker/dependencies/docker.compose.yaml` |
| 4 | `docker/emulator/docker.compose.yaml` |
| 5 | `apps/backend/.env` |
| 6 | `apps/backend/.env.development` |
| 7 | `docker/server/.env.example` |
| 8 | `package.json` |
| 9 | `.devcontainer/devcontainer.json` |
| 10 | `apps/e2e/.env.development` |
| 11 | `.github/workflows/check-prisma-migrations.yaml` |
| 12 | `.github/workflows/docker-server-test.yaml` |
| 13 | `.github/workflows/e2e-api-tests.yaml` |
| 14 | `.github/workflows/e2e-source-of-truth-api-tests.yaml` |
| 15 | `.github/workflows/restart-dev-and-test.yaml` |
| 16 |
`apps/e2e/tests/backend/endpoints/api/v1/internal/email-drafts.test.ts`
|
| 17 | `apps/e2e/tests/backend/endpoints/api/v1/internal/email.test.ts`
|
| 18 | `apps/e2e/tests/backend/endpoints/api/v1/send-email.test.ts` |
| 19 |
`apps/e2e/tests/backend/endpoints/api/v1/unsubscribe-link.test.ts` |
| 20 | `apps/e2e/tests/backend/workflows.test.ts` |
| 21 | `docs/templates/others/self-host.mdx` |
</details>
[](https://discord.gg/n3SsVDAW6U)
[
<!-- RECURSEML_SUMMARY:END -->
<!-- ELLIPSIS_HIDDEN -->
----
> [!IMPORTANT]
> This PR introduces customizable development ports using
`NEXT_PUBLIC_STACK_PORT_PREFIX`, updating configurations, documentation,
and tests accordingly.
>
> - **Behavior**:
> - Default development ports for services are now customizable via
`NEXT_PUBLIC_STACK_PORT_PREFIX`.
> - PostgreSQL port changed from `5432` to
`${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}28`.
> - Inbucket SMTP port changed from `2500` to
`${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}29`.
> - Inbucket POP3 port changed from `1100` to
`${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}30`.
> - OpenTelemetry collector port changed from `4318` to
`${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}31`.
> - **Configuration**:
> - Updated `docker.compose.yaml` to use new port variables for services
like PostgreSQL, Inbucket, and OpenTelemetry.
> - Environment files in `apps/backend`, `apps/dashboard`, and
`apps/e2e` updated to use `NEXT_PUBLIC_STACK_PORT_PREFIX`.
> - `package.json` scripts updated to reflect new port configurations.
> - **Documentation**:
> - Added `CLAUDE-KNOWLEDGE.md` to document new port mappings.
> - Updated `self-host.mdx` to reflect new port configurations.
> - **Testing**:
> - Updated test files in `apps/e2e/tests` to use new port
configurations.
> - Added `helpers/ports.ts` for port-related utilities in tests.
>
> <sup>This description was created by </sup>[<img alt="Ellipsis"
src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=stack-auth%2Fstack-auth&utm_source=github&utm_medium=referral)<sup>
for 76ef55f58f. You can
[customize](https://app.ellipsis.dev/stack-auth/settings/summaries) this
summary. It will automatically update as commits are pushed.</sup>
----
<!-- ELLIPSIS_HIDDEN -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enable configurable development ports via a
NEXT_PUBLIC_STACK_PORT_PREFIX, allowing parallel local environments with
custom port prefixes.
- **Bug Fixes**
- Updated local service port mappings and CI/workflow settings so
tooling and tests use the new prefixed ports consistently.
- **Documentation**
- Added docs and contributor guidance for running multiple parallel
workspaces with custom port prefixes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: N2D4 <N2D4@users.noreply.github.com>
