mirror of
https://github.com/stack-auth/stack.git
synced 2026-07-20 21:29:36 +08:00
clickhouse user sync (#1159)
all-good: Did all the other checks pass? / all-good (push) Has been cancelled
Ensure Prisma migrations are in sync with the schema / check_prisma_migrations (22.x) (push) Has been cancelled
DB migrations are backwards-compatible / Check if migrations changed (push) Has been cancelled
Docker Server Build and Push / Docker Build and Push Server (push) Has been cancelled
Docker Server Build and Run / docker (push) Has been cancelled
Runs E2E API Tests / E2E Tests (Node ${{ matrix.node-version }}, Freestyle ${{ matrix.freestyle-mode }}) (mock, 22.x) (push) Has been cancelled
Runs E2E API Tests / E2E Tests (Node ${{ matrix.node-version }}, Freestyle ${{ matrix.freestyle-mode }}) (prod, 22.x) (push) Has been cancelled
Runs E2E API Tests with custom port prefix / build (22.x) (push) Has been cancelled
Lint & build / lint_and_build (latest) (push) Has been cancelled
Dev Environment Test With Custom Base Port / restart-dev-and-test-with-custom-base-port (push) Has been cancelled
Dev Environment Test / restart-dev-and-test (push) Has been cancelled
Run setup tests with custom base port / setup-tests-with-custom-base-port (push) Has been cancelled
Run setup tests / setup-tests (push) Has been cancelled
TOC Generator / TOC Generator (push) Has been cancelled
DB migrations are backwards-compatible / Test migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migrations are backwards-compatible / No migration changes (skipped) (push) Has been cancelled
all-good: Did all the other checks pass? / all-good (push) Has been cancelled
Ensure Prisma migrations are in sync with the schema / check_prisma_migrations (22.x) (push) Has been cancelled
DB migrations are backwards-compatible / Check if migrations changed (push) Has been cancelled
Docker Server Build and Push / Docker Build and Push Server (push) Has been cancelled
Docker Server Build and Run / docker (push) Has been cancelled
Runs E2E API Tests / E2E Tests (Node ${{ matrix.node-version }}, Freestyle ${{ matrix.freestyle-mode }}) (mock, 22.x) (push) Has been cancelled
Runs E2E API Tests / E2E Tests (Node ${{ matrix.node-version }}, Freestyle ${{ matrix.freestyle-mode }}) (prod, 22.x) (push) Has been cancelled
Runs E2E API Tests with custom port prefix / build (22.x) (push) Has been cancelled
Lint & build / lint_and_build (latest) (push) Has been cancelled
Dev Environment Test With Custom Base Port / restart-dev-and-test-with-custom-base-port (push) Has been cancelled
Dev Environment Test / restart-dev-and-test (push) Has been cancelled
Run setup tests with custom base port / setup-tests-with-custom-base-port (push) Has been cancelled
Run setup tests / setup-tests (push) Has been cancelled
TOC Generator / TOC Generator (push) Has been cancelled
DB migrations are backwards-compatible / Test migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migrations are backwards-compatible / No migration changes (skipped) (push) Has been cancelled
<!-- Make sure you've read the CONTRIBUTING.md guidelines: https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Real-time AI search with project-scoped analytics and dynamic query execution; streaming AI responses replace the placeholder flow. * External DB sync adds ClickHouse support: users sync, sync metadata tracking, tenancy-aware status, and per-mapping throttling. * AI assistant UI shows expandable tool-invocation results and streams via the real AI pipeline. * **Chores** * Dashboard dependencies and workspace exclusions updated; development OpenAI env var added; editor config flag toggled. * **Tests** * E2E coverage extended to validate ClickHouse user sync and analytics queries. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: aadesh18 <[email protected]> Co-authored-by: Konsti Wohlwend <[email protected]>
This commit is contained in:
BilalG1
co-authored by
aadesh18
Konsti Wohlwend
aadesh18
Konsti Wohlwend
parent
6673e63ee7
commit
d09a180dfe
@@ -1,178 +0,0 @@
|
||||
name: Runs E2E API Tests with external source of truth
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
- dev
|
||||
pull_request:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: ${{ github.ref != 'refs/heads/main' && github.ref != 'refs/heads/dev' }}
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubicloud-standard-8
|
||||
env:
|
||||
NODE_ENV: test
|
||||
STACK_ENABLE_HARDCODED_PASSKEY_CHALLENGE_FOR_TESTING: yes
|
||||
STACK_ACCESS_TOKEN_EXPIRATION_TIME: 30m
|
||||
STACK_OVERRIDE_SOURCE_OF_TRUTH: '{"type": "postgres", "connectionString": "postgres://postgres:PASSWORD-PLACEHOLDER--uqfEC1hmmv@localhost:8128/source-of-truth-db?schema=sot-schema"}'
|
||||
STACK_TEST_SOURCE_OF_TRUTH: true
|
||||
STACK_DATABASE_CONNECTION_STRING: "postgres://postgres:PASSWORD-PLACEHOLDER--uqfEC1hmmv@localhost:8128/stackframe"
|
||||
STACK_EXTERNAL_DB_SYNC_MAX_DURATION_MS: "20000"
|
||||
STACK_EXTERNAL_DB_SYNC_DIRECT: "false"
|
||||
|
||||
strategy:
|
||||
matrix:
|
||||
node-version: [22.x]
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup Node.js ${{ matrix.node-version }}
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: ${{ matrix.node-version }}
|
||||
|
||||
- name: Setup pnpm
|
||||
uses: pnpm/action-setup@v4
|
||||
|
||||
# Even just starting the Docker Compose as a daemon is slow because we have to download and build the images
|
||||
# so, we run it in the background
|
||||
- name: Start Docker Compose in background
|
||||
uses: JarvusInnovations/[email protected]
|
||||
with:
|
||||
run: docker compose -f docker/dependencies/docker.compose.yaml up -d &
|
||||
# we don't need to wait on anything, just need to start the daemon
|
||||
wait-on: /dev/null
|
||||
tail: true
|
||||
wait-for: 3s
|
||||
log-output-if: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Create .env.test.local file for apps/backend
|
||||
run: cp apps/backend/.env.development apps/backend/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for apps/dashboard
|
||||
run: cp apps/dashboard/.env.development apps/dashboard/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for apps/e2e
|
||||
run: cp apps/e2e/.env.development apps/e2e/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for docs
|
||||
run: cp docs/.env.development docs/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for examples/cjs-test
|
||||
run: cp examples/cjs-test/.env.development examples/cjs-test/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for examples/demo
|
||||
run: cp examples/demo/.env.development examples/demo/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for examples/docs-examples
|
||||
run: cp examples/docs-examples/.env.development examples/docs-examples/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for examples/e-commerce
|
||||
run: cp examples/e-commerce/.env.development examples/e-commerce/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for examples/middleware
|
||||
run: cp examples/middleware/.env.development examples/middleware/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for examples/supabase
|
||||
run: cp examples/supabase/.env.development examples/supabase/.env.test.local
|
||||
|
||||
- name: Create .env.test.local file for examples/convex
|
||||
run: cp examples/convex/.env.development examples/convex/.env.test.local
|
||||
|
||||
- name: Build
|
||||
run: pnpm build
|
||||
|
||||
- name: Wait on Postgres
|
||||
run: pnpm run wait-until-postgres-is-ready:pg_isready
|
||||
|
||||
- name: Wait on Inbucket
|
||||
run: pnpx wait-on tcp:localhost:8129
|
||||
|
||||
- name: Wait on Svix
|
||||
run: pnpx wait-on tcp:localhost:8113
|
||||
|
||||
- name: Wait on QStash
|
||||
run: pnpx wait-on tcp:localhost:8125
|
||||
|
||||
- name: Create source-of-truth database and schema
|
||||
run: |
|
||||
psql postgres://postgres:PASSWORD-PLACEHOLDER--uqfEC1hmmv@localhost:8128/postgres -c "CREATE DATABASE \"source-of-truth-db\";"
|
||||
psql postgres://postgres:PASSWORD-PLACEHOLDER--uqfEC1hmmv@localhost:8128/source-of-truth-db -c "CREATE SCHEMA \"sot-schema\";"
|
||||
|
||||
- name: Initialize database
|
||||
run: pnpm run db:init
|
||||
|
||||
- name: Start stack-backend in background
|
||||
uses: JarvusInnovations/[email protected]
|
||||
with:
|
||||
run: pnpm run start:backend --log-order=stream &
|
||||
wait-on: |
|
||||
http://localhost:8102
|
||||
tail: true
|
||||
wait-for: 30s
|
||||
log-output-if: true
|
||||
- name: Start stack-dashboard in background
|
||||
uses: JarvusInnovations/[email protected]
|
||||
with:
|
||||
run: pnpm run start:dashboard --log-order=stream &
|
||||
wait-on: |
|
||||
http://localhost:8101
|
||||
tail: true
|
||||
wait-for: 30s
|
||||
log-output-if: true
|
||||
- name: Start mock-oauth-server in background
|
||||
uses: JarvusInnovations/[email protected]
|
||||
with:
|
||||
run: pnpm run start:mock-oauth-server --log-order=stream &
|
||||
wait-on: |
|
||||
http://localhost:8102
|
||||
tail: true
|
||||
wait-for: 30s
|
||||
log-output-if: true
|
||||
- name: Start run-email-queue in background
|
||||
uses: JarvusInnovations/[email protected]
|
||||
with:
|
||||
run: pnpm -C apps/backend run run-email-queue --log-order=stream &
|
||||
wait-on: |
|
||||
http://localhost:8102
|
||||
tail: true
|
||||
wait-for: 30s
|
||||
log-output-if: true
|
||||
- name: Start run-cron-jobs in background
|
||||
uses: JarvusInnovations/[email protected]
|
||||
with:
|
||||
run: pnpm -C apps/backend run run-cron-jobs --log-order=stream &
|
||||
wait-on: |
|
||||
http://localhost:8102
|
||||
tail: true
|
||||
wait-for: 30s
|
||||
log-output-if: true
|
||||
|
||||
- name: Wait 10 seconds
|
||||
run: sleep 10
|
||||
|
||||
- name: Run tests
|
||||
run: pnpm test run --exclude "**/external-db-sync*.test.ts" # external-db-sync does not support external sot
|
||||
|
||||
- name: Run tests again (attempt 1)
|
||||
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev'
|
||||
run: pnpm test run --exclude "**/external-db-sync*.test.ts"
|
||||
|
||||
- name: Run tests again (attempt 2)
|
||||
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev'
|
||||
run: pnpm test run --exclude "**/external-db-sync*.test.ts"
|
||||
|
||||
- name: Verify data integrity
|
||||
run: pnpm run verify-data-integrity --no-bail
|
||||
|
||||
- name: Print Docker Compose logs
|
||||
if: always()
|
||||
run: docker compose -f docker/dependencies/docker.compose.yaml logs
|
||||
Reference in New Issue
Block a user