wait-for-deps used to loop forever on each service, so any single
dep that failed to start (e.g. a service crash-looping under TCG)
hung the build until the outer 6000s provision timeout.
Rewrite as a wait_for helper with:
- Hard 1500s budget across the full dep wait (overridable via
STACK_DEPS_TIMEOUT). On timeout, dump docker ps -a, last 300 lines
of the deps container, and per-service reachability, then exit 1
so provision-build's cleanup trap fires and the VM shuts down fast.
- "<service> ready (Ns)" log lines on each service so successful
runs show which service was the bottleneck.
- 30s heartbeat per service so long-running waits don't look frozen.
amd64 is unaffected — services come up in ~1s each under KVM, which
is well inside any threshold here.
The previous commit set NODE_OPTIONS=--jitless on the migration
docker exec. That was wrong for two reasons:
- --jitless disables eval and new Function, which some code in the
migration path uses, so it broke amd64 builds that had been passing.
- --jitless is a V8 feature gate, not a TCG workaround. If it breaks
one arch it breaks both — it could never have helped arm64 either.
Revert the --jitless flag and rely on -cpu cortex-a72 (added in the
parent commit) as the root-cause fix for the arm64 TCG SIGTRAP.
Keep the stdout/stderr capture for the migration exec so the next
failure dumps the actual node error through log-provision instead of
being swallowed by the serial-only stream.
The arm64 matrix entry cross-compiles on the amd64 CI runner, so the
guest runs under QEMU TCG. Under -cpu max, V8 emits armv8.5+ JIT code
that TCG mistranslates and node crashes with SIGTRAP (exit 133)
during migrations. Three changes together get it working:
- Drop to -cpu cortex-a72 for TCG arm64 guests. Limits V8 to
armv8.0-a which TCG handles cleanly. Native paths (HVF/KVM) keep
-cpu max for full performance.
- Run migrations with NODE_OPTIONS=--jitless as belt-and-suspenders.
Migrations are I/O-bound so the perf hit is negligible.
- Skip the in-guest smoke test on arm64. A full Next.js backend under
cross-arch TCG either SIGTRAPs or times out; the amd64 build still
runs the smoke test, which covers every non-arch-specific code
path. Arch is propagated into the guest via a new build-arch.env
marker in the stack-bundle ISO.
Provisioning used to silently wait out the full 6000s timeout on any
guest-side failure because the cleanup trap only logged the error. Now
it writes STACK_CLOUD_INIT_FAILED and shuts the VM down, and the host
waiter breaks on that marker and reports it distinctly.
Also bump smoke test timeout 120s->300s, dump docker ps / container
logs / free -m / verbose curl when it fails, log the qemu accel path,
and enable /dev/kvm on the CI runner so the VM isn't stuck in TCG.
The custom-base-port and db-migration-backwards-compatibility workflows
were running cron jobs with `with-env:dev` instead of `with-env:test`,
causing ClickHouse sync mismatches in verify-data-integrity.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Streamlined CI test workflows to standardize background cron job
startup for more consistent test runs.
* **Tests**
* Improved end-to-end test reliability by aligning background process
behavior across suites.
* **Bug Fixes**
* Enhanced data verification reliability by ensuring external database
sync before integrity checks and tightening comparison ordering for
certain records, reducing false mismatch detections.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Normalize empty route path to / for valid OpenAPI path keys
- Drop invalid OAS3 top-level type on header parameters
- Write client/server/admin/webhooks JSON to docs-mintlify/openapi on codegen
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* External DB sync now covers teams, team members, permissions,
invitations, email outbox, session replays, refresh tokens, and
connected accounts.
* New sequence ID fields and automatic change-flagging added to many
record types to enable incremental sync.
* **Improvements**
* Added concurrent indexes, faster/parallelized sync pipelines,
verification tooling, and richer observability.
* Dashboard sequencer stats expanded and end-to-end sync tests
significantly extended.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
s3mock 5.0.0 was released today and latest now points to it. The major
version bump breaks initialBuckets env var handling, causing
NoSuchBucket errors in CI.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Pinned service dependency to a specific version for improved
deployment consistency and stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Performance Improvements**
* Redesigned AI request processing to operate efficiently at the backend
level, providing faster responses and improved reliability for
AI-powered features
* **Enhanced AI Capabilities**
* Improved model validation and selection with explicit allowlist of
supported models, enabling broader model availability and better feature
stability
* **Developer Updates**
* Refreshed local development environment configuration
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Temporarily repeat the cross-subdomain cookie test 10 times to verify CI
stability. Increased waitUntil timeout from 10s to 30s to account for
network retry exponential backoff.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
# Release Notes
This pull request contains internal testing infrastructure updates only.
A test timeout value was adjusted to improve test reliability. **No
user-visible changes or new features are included in this release.**
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
### Context
The `emails-low-level.tsx` file is where the sending logic of an email
over smtp occurs. It tries to send an email, and if there is an issue it
has logic to mark the email as retryable or not. Then, the
`email-queue-step` itself handles the actual retry efforts based on this
flag and a few other factors (like if max retries have been attempted).
Currently, if an error code isn't explicitly accounted for, it falls
back to being an "unknown error while sending" and is marked as not
retryable.
However, SMTP [defines the 4yz class of error codes as being
retryable](https://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml).
### Summary of Changes
We add fallback logic at the end of the `responseCode` specific block to
mark the 4yz class of error codes as retryable. While the message here
is less specific, that is okay since any codes we want to explicitly
account for are handled above it. The main thing here is that these
should be retried.
## Summary
- add a public `defineStackConfig` helper and `StackConfig` type for
nested config authoring
- emit helper-based nested config files from the CLI and local emulator
- update type coverage and e2e expectations for the new `stack.config`
format
## Testing
- pnpm --filter ./packages/stack-shared typecheck
- pnpm --filter ./packages/stack-cli typecheck
- pnpm --filter ./apps/backend typecheck
- pnpm --filter ./apps/e2e typecheck
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Type-safe configuration API with compile-time validation
* New config rendering utility for producing typed config files
* Public local-emulator settings and a public helper to detect emulator
mode
* Added --overwrite flag for config pull
* **Improvements**
* Stronger validation and clearer errors for invalid or conflicting
config shapes
* Config output now includes explicit TypeScript typing
* **Tests**
* Added and strengthened tests for config authoring, rendering, CLI
behavior, and emulator flows
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
commit 5d43722575b826a8ed8dbb6b828f48eae4bca02c
Author: mantrakp04 <mantrakp@gmail.com>
Date: Wed Mar 18 12:27:01 2026 -0700
Add QEMU emulator snapshot functionality and reset command
- Introduced a new `emulator-qemu:reset` command in package.json to
clear snapshots and force a fresh boot of the emulator.
- Enhanced the `run-emulator.sh` script to support saving and restoring
snapshots, significantly reducing restart time from ~62s to ~4s.
- Implemented logic to check for existing snapshots and restore them
during startup, improving the emulator's efficiency.
- Updated documentation in CLAUDE-KNOWLEDGE.md to explain the new
snapshot restore process and its benefits.
These changes enhance the QEMU emulator's performance and usability for
developers, providing a more efficient workflow during development.
commit 3877445bdd83cb8690da18c8520bf260d2795172
Author: mantrakp04 <mantrakp@gmail.com>
Date: Wed Mar 18 11:55:18 2026 -0700
Enhance QEMU emulator performance and configuration management
- Added optimizations to the QEMU emulator's app container startup
process, reducing startup time from ~92s to ~62s by using qcow2 backing
files and setting the working directory to /app.
- Updated the build-image.sh script to conditionally wait for background
processes, improving robustness.
- Modified the run-emulator.sh script to create the disk image using
qcow2 format instead of copying, enhancing efficiency.
- Adjusted the cloud-init user-data to set STACK_RUNTIME_WORK_DIR to
/app, streamlining file operations during container initialization.
- Improved the entrypoint script to avoid unnecessary file copying when
the working directory is set to /app.
These changes significantly enhance the performance and usability of the
QEMU emulator for developers.
commit e0b86d3f1d5c08e46d0d343bc632e2a8c5777845
Author: mantrakp04 <mantrakp@gmail.com>
Date: Wed Mar 18 11:07:55 2026 -0700
Refactor local emulator configuration management and enhance Docker
setup
- Removed redundant comments and improved code clarity in the local
emulator's route handling.
- Streamlined the Dockerfile and docker-compose.yaml for better
readability and maintenance.
- Updated entrypoint and initialization scripts to enhance service
startup processes.
- Introduced a new common script for QEMU emulator to centralize
architecture detection and firmware handling.
- Enhanced error handling in the host file bridge for improved
robustness.
- Removed obsolete country code utilities to clean up the codebase.
These changes significantly improve the local emulator's configuration
management and overall setup experience for developers.
commit 4fb0f93c6cc4f749a14acf0228c261e180875609
Author: mantrakp04 <mantrakp@gmail.com>
Date: Wed Mar 18 10:24:53 2026 -0700
Implement local emulator file bridge for enhanced configuration
management
- Introduced a new host file bridge to facilitate reading and writing
configuration files between the local emulator and the host system.
- Refactored the local-emulator module to utilize the file bridge for
file operations, improving error handling and response validation.
- Added tests to ensure the file bridge functionality works as expected,
including handling of non-existent files and writing configurations.
- Updated the run-emulator script to start the file bridge
automatically, ensuring seamless integration during emulator startup.
- Enhanced documentation to reflect the new file bridge capabilities and
usage instructions.
These changes significantly improve the local emulator's ability to
manage configuration files, enhancing the development experience.
commit 3d18a7ce5bbf00a62a40a3f48f27856e79ecc62f
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 22:36:46 2026 -0700
Refactor QEMU local emulator setup and enhance app bundle handling
- Introduced a new script for packaging Docker images into a compressed
app bundle, improving the emulator's deployment process.
- Updated build-image.sh to create a runtime configuration ISO, ensuring
better management of environment settings.
- Enhanced cloud-init user-data scripts for both dev-server and deps
guests, streamlining service setup and configuration.
- Improved the run-emulator.sh script to facilitate better handling of
runtime configurations and dependencies.
- Adjusted the .gitignore to include .DS_Store and removed obsolete
entries, cleaning up the repository.
These changes significantly enhance the local emulator's functionality
and reliability for developers.
commit 8a35fb1ce79898d73e2259e256c11b6fd9b0a584
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 21:52:24 2026 -0700
Enhance local emulator functionality and configuration
- Updated package.json to improve the start-emulator command, providing
clearer dashboard and backend URLs.
- Added a new wait-until-emulator-is-ready command to ensure the
emulator is fully operational before proceeding.
- Refactored the local-emulator project route to streamline file
existence checks and default config creation.
- Enhanced user guidance in the dashboard for local Stack config file
handling.
- Updated tests to reflect changes in config file handling, ensuring
non-existent files are created with default settings.
- Improved Docker configurations for the local emulator, including new
environment variables and service dependencies.
These changes significantly enhance the local development experience and
emulator reliability.
commit 3910ed4bc40bbb37340c1c316c24c2826ba372bd
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 19:59:36 2026 -0700
Remove unused stash-0.patch file to clean up the repository.
commit 74146d974458037a7a9590120a524629a1a6a162
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 19:58:46 2026 -0700
Enhance QEMU local emulator with app bundle support and runtime
configuration
- Introduced a new script to package the backend and dashboard assets
into a standalone app bundle for the QEMU emulator.
- Updated the build-image.sh script to create an ISO containing the app
bundle, ensuring the guest image includes the full runtime.
- Modified cloud-init user-data to handle the new app bundle and runtime
configuration, improving the setup process for local development.
- Enhanced the run-emulator.sh script to prepare and mount the runtime
configuration ISO, facilitating better environment management for the
emulator.
- Updated the user-data to include necessary environment variables for
the stack application, ensuring seamless integration during startup.
These changes significantly improve the local emulator's functionality
and ease of use for developers.
commit 9e865a1cf524398bc58f00e0836278775c4ae936
Author: mantrakp04 <mantrakp@gmail.com>
Date: Tue Mar 17 16:50:45 2026 -0700
Enhance local emulator setup with new services and configurations
- Added Docker support for a local emulator, integrating PostgreSQL,
Redis, Inbucket, Svix, ClickHouse, MinIO, and QStash.
- Introduced new scripts for managing the emulator lifecycle, including
build and run commands.
- Implemented cloud-init provisioning for automatic service setup on
first boot.
- Updated package.json with new commands for emulator management and
added dotenv-cli for environment variable management.
- Added tests for OAuth authorization flow to return JSON responses.
- Included configuration files for ClickHouse and user management.
This commit significantly improves the local development experience by
providing a comprehensive emulator environment.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced a local QEMU-based emulator for development with bundled
services (PostgreSQL, Redis, ClickHouse, MinIO, Inbucket, Svix, QStash).
* Added CLI commands to manage the emulator (start, stop, reset, status,
pull images).
* Added emulator status dashboard to monitor service health.
* Introduced new configuration system via `stack.config.ts`.
* **Tests**
* Added configuration read/write tests for the emulator.
* Added emulator CLI validation tests.
* **Documentation**
* Added emulator setup and usage guide.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Cross-subdomain refresh cookies were not being deleted correctly because
the domain option was not passed to deleteCookie/deleteCookieClient.
This caused stale cookies to accumulate and auth state to persist across
subdomains after sign-out.
Also eagerly warms the trusted parent domain cache on app construction
to avoid a race condition where navigation after sign-in could prevent
the cross-subdomain cookie from being written.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Automatically recreates a missing cross-subdomain refresh cookie on
app startup in browser sessions when applicable.
* **Bug Fixes**
* Cookie deletions now correctly scope removals to the encoded parent
domain when applicable for both browser and server token-store flows.
* **Performance**
* Pre-warms a domain-resolution cache in browser token-store scenarios
to reduce authentication latency.
* **Tests**
* Added end-to-end tests validating custom refresh-cookie name
encoding/decoding, non-custom cookie handling, and eager cookie
recreation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
