emu with a q stuff (#1266)

commit 5d43722575b826a8ed8dbb6b828f48eae4bca02c
Author: mantrakp04 <[email protected]>
Date:   Wed Mar 18 12:27:01 2026 -0700

    Add QEMU emulator snapshot functionality and reset command

- Introduced a new `emulator-qemu:reset` command in package.json to
clear snapshots and force a fresh boot of the emulator.
- Enhanced the `run-emulator.sh` script to support saving and restoring
snapshots, significantly reducing restart time from ~62s to ~4s.
- Implemented logic to check for existing snapshots and restore them
during startup, improving the emulator's efficiency.
- Updated documentation in CLAUDE-KNOWLEDGE.md to explain the new
snapshot restore process and its benefits.

These changes enhance the QEMU emulator's performance and usability for
developers, providing a more efficient workflow during development.

commit 3877445bdd83cb8690da18c8520bf260d2795172
Author: mantrakp04 <[email protected]>
Date:   Wed Mar 18 11:55:18 2026 -0700

    Enhance QEMU emulator performance and configuration management

- Added optimizations to the QEMU emulator's app container startup
process, reducing startup time from ~92s to ~62s by using qcow2 backing
files and setting the working directory to /app.
- Updated the build-image.sh script to conditionally wait for background
processes, improving robustness.
- Modified the run-emulator.sh script to create the disk image using
qcow2 format instead of copying, enhancing efficiency.
- Adjusted the cloud-init user-data to set STACK_RUNTIME_WORK_DIR to
/app, streamlining file operations during container initialization.
- Improved the entrypoint script to avoid unnecessary file copying when
the working directory is set to /app.

These changes significantly enhance the performance and usability of the
QEMU emulator for developers.

commit e0b86d3f1d5c08e46d0d343bc632e2a8c5777845
Author: mantrakp04 <[email protected]>
Date:   Wed Mar 18 11:07:55 2026 -0700

Refactor local emulator configuration management and enhance Docker
setup

- Removed redundant comments and improved code clarity in the local
emulator's route handling.
- Streamlined the Dockerfile and docker-compose.yaml for better
readability and maintenance.
- Updated entrypoint and initialization scripts to enhance service
startup processes.
- Introduced a new common script for QEMU emulator to centralize
architecture detection and firmware handling.
- Enhanced error handling in the host file bridge for improved
robustness.
    - Removed obsolete country code utilities to clean up the codebase.

These changes significantly improve the local emulator's configuration
management and overall setup experience for developers.

commit 4fb0f93c6cc4f749a14acf0228c261e180875609
Author: mantrakp04 <[email protected]>
Date:   Wed Mar 18 10:24:53 2026 -0700

Implement local emulator file bridge for enhanced configuration
management

- Introduced a new host file bridge to facilitate reading and writing
configuration files between the local emulator and the host system.
- Refactored the local-emulator module to utilize the file bridge for
file operations, improving error handling and response validation.
- Added tests to ensure the file bridge functionality works as expected,
including handling of non-existent files and writing configurations.
- Updated the run-emulator script to start the file bridge
automatically, ensuring seamless integration during emulator startup.
- Enhanced documentation to reflect the new file bridge capabilities and
usage instructions.

These changes significantly improve the local emulator's ability to
manage configuration files, enhancing the development experience.

commit 3d18a7ce5bbf00a62a40a3f48f27856e79ecc62f
Author: mantrakp04 <[email protected]>
Date:   Tue Mar 17 22:36:46 2026 -0700

    Refactor QEMU local emulator setup and enhance app bundle handling

- Introduced a new script for packaging Docker images into a compressed
app bundle, improving the emulator's deployment process.
- Updated build-image.sh to create a runtime configuration ISO, ensuring
better management of environment settings.
- Enhanced cloud-init user-data scripts for both dev-server and deps
guests, streamlining service setup and configuration.
- Improved the run-emulator.sh script to facilitate better handling of
runtime configurations and dependencies.
- Adjusted the .gitignore to include .DS_Store and removed obsolete
entries, cleaning up the repository.

These changes significantly enhance the local emulator's functionality
and reliability for developers.

commit 8a35fb1ce79898d73e2259e256c11b6fd9b0a584
Author: mantrakp04 <[email protected]>
Date:   Tue Mar 17 21:52:24 2026 -0700

    Enhance local emulator functionality and configuration

- Updated package.json to improve the start-emulator command, providing
clearer dashboard and backend URLs.
- Added a new wait-until-emulator-is-ready command to ensure the
emulator is fully operational before proceeding.
- Refactored the local-emulator project route to streamline file
existence checks and default config creation.
- Enhanced user guidance in the dashboard for local Stack config file
handling.
- Updated tests to reflect changes in config file handling, ensuring
non-existent files are created with default settings.
- Improved Docker configurations for the local emulator, including new
environment variables and service dependencies.

These changes significantly enhance the local development experience and
emulator reliability.

commit 3910ed4bc40bbb37340c1c316c24c2826ba372bd
Author: mantrakp04 <[email protected]>
Date:   Tue Mar 17 19:59:36 2026 -0700

    Remove unused stash-0.patch file to clean up the repository.

commit 74146d974458037a7a9590120a524629a1a6a162
Author: mantrakp04 <[email protected]>
Date:   Tue Mar 17 19:58:46 2026 -0700

Enhance QEMU local emulator with app bundle support and runtime
configuration

- Introduced a new script to package the backend and dashboard assets
into a standalone app bundle for the QEMU emulator.
- Updated the build-image.sh script to create an ISO containing the app
bundle, ensuring the guest image includes the full runtime.
- Modified cloud-init user-data to handle the new app bundle and runtime
configuration, improving the setup process for local development.
- Enhanced the run-emulator.sh script to prepare and mount the runtime
configuration ISO, facilitating better environment management for the
emulator.
- Updated the user-data to include necessary environment variables for
the stack application, ensuring seamless integration during startup.

These changes significantly improve the local emulator's functionality
and ease of use for developers.

commit 9e865a1cf524398bc58f00e0836278775c4ae936
Author: mantrakp04 <[email protected]>
Date:   Tue Mar 17 16:50:45 2026 -0700

    Enhance local emulator setup with new services and configurations

- Added Docker support for a local emulator, integrating PostgreSQL,
Redis, Inbucket, Svix, ClickHouse, MinIO, and QStash.
- Introduced new scripts for managing the emulator lifecycle, including
build and run commands.
- Implemented cloud-init provisioning for automatic service setup on
first boot.
- Updated package.json with new commands for emulator management and
added dotenv-cli for environment variable management.
    - Added tests for OAuth authorization flow to return JSON responses.
    - Included configuration files for ClickHouse and user management.

This commit significantly improves the local development experience by
providing a comprehensive emulator environment.

<!--

Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md

-->


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **New Features**
* Introduced a local QEMU-based emulator for development with bundled
services (PostgreSQL, Redis, ClickHouse, MinIO, Inbucket, Svix, QStash).
* Added CLI commands to manage the emulator (start, stop, reset, status,
pull images).
  * Added emulator status dashboard to monitor service health.
  * Introduced new configuration system via `stack.config.ts`.

* **Tests**
  * Added configuration read/write tests for the emulator.
  * Added emulator CLI validation tests.

* **Documentation**
  * Added emulator setup and usage guide.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

.github/workflows/qemu-emulator-build.yaml

@@ -0,0 +1,255 @@
+name: Build & Publish QEMU Emulator Images
+
+on:
+  push:
+    branches:
+      - main
+      - dev
+  pull_request:
+    paths:
+      - 'docker/local-emulator/**'
+      - '.github/workflows/qemu-emulator-build.yaml'
+  workflow_dispatch:
+    inputs:
+      publish:
+        description: 'Publish images to GitHub Releases'
+        type: boolean
+        default: false
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.ref != 'refs/heads/main' && github.ref != 'refs/heads/dev' }}
+
+env:
+  EMULATOR_IMAGE_NAME: stack-local-emulator
+
+jobs:
+  build:
+    name: Build QEMU Image (${{ matrix.arch }})
+    runs-on: ubicloud-standard-8
+    timeout-minutes: 120
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - arch: amd64
+          - arch: arm64
+
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Set up QEMU user-mode emulation
+        uses: docker/setup-qemu-action@v3
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Install QEMU dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y qemu-system-x86 qemu-system-arm qemu-utils genisoimage socat qemu-efi-aarch64
+
+      - name: Build QEMU image
+        run: |
+          chmod +x docker/local-emulator/qemu/build-image.sh
+          EMULATOR_PROVISION_TIMEOUT=6000 \
+            docker/local-emulator/qemu/build-image.sh ${{ matrix.arch }}
+
+      - name: Generate emulator env
+        run: node docker/local-emulator/generate-env-development.mjs
+
+      - name: Start emulator and verify
+        run: |
+          chmod +x docker/local-emulator/qemu/run-emulator.sh
+          EMULATOR_ARCH=${{ matrix.arch }} \
+          EMULATOR_READY_TIMEOUT=3200 \
+            docker/local-emulator/qemu/run-emulator.sh start
+
+      - name: Verify services are healthy
+        run: |
+          EMULATOR_ARCH=${{ matrix.arch }} \
+            docker/local-emulator/qemu/run-emulator.sh status
+
+      - name: Stop emulator
+        if: always()
+        run: |
+          EMULATOR_ARCH=${{ matrix.arch }} \
+            docker/local-emulator/qemu/run-emulator.sh stop
+
+      - name: Package image
+        run: |
+          BASE_IMG="docker/local-emulator/qemu/images/stack-emulator-${{ matrix.arch }}.qcow2"
+          cp "$BASE_IMG" "stack-emulator-${{ matrix.arch }}.qcow2"
+
+      - name: Upload image artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: qemu-emulator-${{ matrix.arch }}
+          path: stack-emulator-${{ matrix.arch }}.qcow2
+          retention-days: 30
+          compression-level: 0
+
+  test:
+    name: Smoke Test (${{ matrix.arch }})
+    needs: build
+    runs-on: ubicloud-standard-8
+    timeout-minutes: 60
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - arch: amd64
+
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install QEMU dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y qemu-system-x86 qemu-utils genisoimage socat
+
+      - name: Download built image
+        uses: actions/download-artifact@v4
+        with:
+          name: qemu-emulator-${{ matrix.arch }}
+          path: docker/local-emulator/qemu/images/
+
+      - name: Generate emulator env
+        run: node docker/local-emulator/generate-env-development.mjs
+
+      - name: Start emulator from artifact
+        run: |
+          chmod +x docker/local-emulator/qemu/run-emulator.sh docker/local-emulator/qemu/common.sh
+          EMULATOR_ARCH=${{ matrix.arch }} \
+          EMULATOR_READY_TIMEOUT=600 \
+            docker/local-emulator/qemu/run-emulator.sh start
+
+      - name: Verify services are healthy
+        run: |
+          EMULATOR_ARCH=${{ matrix.arch }} \
+            docker/local-emulator/qemu/run-emulator.sh status
+
+      - name: Smoke test — backend health
+        run: curl -sf http://localhost:26701/health?db=1
+
+      - name: Smoke test — dashboard reachable
+        run: curl -sf -o /dev/null -w "HTTP %{http_code}\n" http://localhost:26700/handler/sign-in
+
+      - name: Smoke test — MinIO health
+        run: curl -sf http://localhost:26702/minio/health/live
+
+      - name: Smoke test — Inbucket reachable
+        run: curl -sf -o /dev/null -w "HTTP %{http_code}\n" http://localhost:26703/
+
+      - name: Stop emulator
+        if: always()
+        run: |
+          EMULATOR_ARCH=${{ matrix.arch }} \
+            docker/local-emulator/qemu/run-emulator.sh stop
+
+      - name: Print serial log on failure
+        if: failure()
+        run: tail -100 docker/local-emulator/qemu/run/vm/serial.log 2>/dev/null || true
+
+  publish:
+    name: Publish to GitHub Releases
+    needs: [build, test]
+    if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev' || (github.event_name == 'workflow_dispatch' && inputs.publish)
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Download all artifacts
+        uses: actions/download-artifact@v4
+        with:
+          path: artifacts
+
+      - name: Prepare release assets
+        run: |
+          mkdir -p release
+          SHORT_SHA="${GITHUB_SHA:0:8}"
+          BRANCH="${GITHUB_REF_NAME}"
+          DATE="$(date -u +%Y%m%d)"
+          TAG="emulator-${BRANCH}-${DATE}-${SHORT_SHA}"
+          echo "RELEASE_TAG=${TAG}" >> "$GITHUB_ENV"
+          echo "SHORT_SHA=${SHORT_SHA}" >> "$GITHUB_ENV"
+
+          for f in artifacts/qemu-emulator-*/*.qcow2; do
+            cp "$f" release/
+          done
+
+          cat > release-notes.md <<EOF
+          ## QEMU Emulator Images
+
+          Built from \`${BRANCH}\` @ \`${GITHUB_SHA}\`
+
+          ### Images
+          | File | Description |
+          |------|-------------|
+          | \`stack-emulator-arm64.qcow2\` | ARM64 emulator image |
+          | \`stack-emulator-amd64.qcow2\` | AMD64 emulator image |
+
+          ### Usage
+          \`\`\`bash
+          stack emulator pull
+          stack emulator start
+          \`\`\`
+          EOF
+
+          ls -lh release/
+
+      - name: Create or update GitHub Release
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          TITLE="QEMU Emulator — ${{ github.ref_name }} ($SHORT_SHA)"
+
+          if gh release view "$RELEASE_TAG" >/dev/null 2>&1; then
+            gh release edit "$RELEASE_TAG" \
+              --title "$TITLE" \
+              --notes-file release-notes.md \
+              --prerelease
+            gh release upload "$RELEASE_TAG" release/* --clobber
+          else
+            gh release create "$RELEASE_TAG" \
+              --title "$TITLE" \
+              --notes-file release-notes.md \
+              --prerelease \
+              release/*
+          fi
+
+      - name: Update latest tag for branch
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          LATEST_TAG="emulator-${{ github.ref_name }}-latest"
+          TITLE="QEMU Emulator — ${{ github.ref_name }} (latest)"
+          NOTES="Latest emulator images from \`${{ github.ref_name }}\`. Auto-updated on each build."
+
+          if gh release view "$LATEST_TAG" >/dev/null 2>&1; then
+            gh release edit "$LATEST_TAG" \
+              --draft \
+              --prerelease \
+              --target "${{ github.sha }}" \
+              --title "$TITLE" \
+              --notes "$NOTES"
+          else
+            gh release create "$LATEST_TAG" \
+              --draft \
+              --prerelease \
+              --target "${{ github.sha }}" \
+              --title "$TITLE" \
+              --notes "$NOTES" \
+            || gh release edit "$LATEST_TAG" \
+              --draft \
+              --prerelease \
+              --target "${{ github.sha }}" \
+              --title "$TITLE" \
+              --notes "$NOTES"
+          fi
+
+          gh release upload "$LATEST_TAG" release/* --clobber
+          gh release edit "$LATEST_TAG" --draft=false --prerelease