### Object of this PR
This PR is NOT a monolithic series of fixes for the payments suite + a
complete rework. Its aims were
a) introducing and robustly testing the bulldozer db system
b) reworking the payments underlying architecture to use bulldozer for
correctness and scalability
c) Achieving parity with the old payments system excepting a few changes
like ensuring correctness of the ledger algo
There may still be some work to do with handling refunds, decoupling the
concepts of purchases from that of products, and some other things.
### Ledger Algorithm
This has been tuned and fixed. Item removals i.e negative item quantity
changes will apply to the soonest expiring item grant i.e positive item
quantity change. This is what is best for the user. Item grants can also
expire, and when they expire we obviate whatever is left of their
original capacity (meaning after all the removals that were applied to
it). Our ledger algo is applied via Bulldozer, so automatic
re-computation is handled when a new grant/ removal is inserted in the
middle of the existing ones.
### Things we got rid of
* No more automatic support for default products. You can use $0 plan
provisions to accomplish the same effect but it's manual
* Negative item quantity changes (i.e item removals) no longer can have
expiries
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced payment processing pipeline with improved data consistency
and state management.
* Advanced refund handling with comprehensive transaction tracking.
* Better tracking and management of customer item quantities and owned
products.
* Improved subscription lifecycle management including period-end
handling.
* **Bug Fixes**
* Fixed payment data integrity verification.
* Improved handling of edge cases in refund scenarios.
* **Chores**
* Updated cSpell configuration with additional words.
* Expanded developer documentation for linting workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
Co-authored-by: Aadesh Kheria <kheriaaadesh@gmail.com>
Co-authored-by: Mantra <87142457+mantrakp04@users.noreply.github.com>
- Updated package versions for '@supabase/*' libraries to 2.99.2 and
'@supabase/ssr' to 0.9.0.
- Added new devDependencies for 'rimraf' and 'framer-motion' in the
pnpm-lock file.
- Modified Next.js configuration to conditionally omit 'X-Frame-Options'
in development mode for better integration with Stack Auth dev tools.
- Refactored component exports in the template package to include
tracking for dev tools.
- Introduced new dev tool components and context for improved logging
and state management.
- Added styles for the dev tool indicator and panel, ensuring a
consistent dark theme.
- Implemented fetch interception to log API calls and user
authentication events in the dev tool.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added comprehensive Developer Tools interface with tabs for Overview,
Components, AI Chat, Console, Dashboard, and Support.
* Integrated AI Chat assistant within Developer Tools for enhanced
debugging.
* Added component version tracking and update notifications.
* Implemented API request logging and event monitoring.
* Enhanced feedback system with support for bug reports and feature
requests.
* **Bug Fixes**
* Fixed Content Security Policy headers for local development
environments.
* **Dependencies**
* Added AI SDK integration packages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
- Added support for `@opentelemetry/sdk-node` in the backend.
- Updated various dependencies including AWS SDK and OpenTelemetry
packages.
- Implemented graceful shutdown handling for non-Vercel runtimes in
`prisma-client.tsx`.
- Enhanced AWS credentials retrieval to support GCP Workload Identity
Federation.
- Introduced a Dockerfile for Cloud Run deployment, optimizing the
backend build process.
- Updated `.gitignore` to include Terraform runtime files and secrets.
This commit improves the backend's observability and deployment
flexibility, particularly for Cloud Run environments.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* OpenTelemetry observability with dynamic provider selection per
deployment.
* Cloud Run trusted-proxy support for accurate client IP handling.
* Graceful shutdown that waits for in-flight background work.
* New background-task handling to improve async webhook/email delivery
reliability.
* AWS credential providers added (Vercel OIDC & GCP Workload Identity
Federation).
* Dockerized backend image for Cloud Run / self-host deployments.
* **Chores**
* Updated dependencies for OpenTelemetry and AWS SDK support.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* External DB sync now covers teams, team members, permissions,
invitations, email outbox, session replays, refresh tokens, and
connected accounts.
* New sequence ID fields and automatic change-flagging added to many
record types to enable incremental sync.
* **Improvements**
* Added concurrent indexes, faster/parallelized sync pipelines,
verification tooling, and richer observability.
* Dashboard sequencer stats expanded and end-to-end sync tests
significantly extended.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
- add a public `defineStackConfig` helper and `StackConfig` type for
nested config authoring
- emit helper-based nested config files from the CLI and local emulator
- update type coverage and e2e expectations for the new `stack.config`
format
## Testing
- pnpm --filter ./packages/stack-shared typecheck
- pnpm --filter ./packages/stack-cli typecheck
- pnpm --filter ./apps/backend typecheck
- pnpm --filter ./apps/e2e typecheck
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Type-safe configuration API with compile-time validation
* New config rendering utility for producing typed config files
* Public local-emulator settings and a public helper to detect emulator
mode
* Added --overwrite flag for config pull
* **Improvements**
* Stronger validation and clearer errors for invalid or conflicting
config shapes
* Config output now includes explicit TypeScript typing
* **Tests**
* Added and strengthened tests for config authoring, rendering, CLI
behavior, and emulator flows
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Updated `validateVerifyResponse` to capture errors for invalid or
unexpected responses.
- Improved handling of malformed responses in `checkEmailWithEmailable`,
ensuring a consistent return structure.
- Refactored `getDerivedSignUpCountryCode` to log errors for non-ISO
country codes.
- Simplified country code determination logic in
`createOrUpgradeAnonymousUserWithRules`.
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Replaced country code dropdown selection with a direct text input
field for simpler data entry.
* Updated country code validation to accept any 2-letter code format,
improving flexibility.
* **Bug Fixes**
* Refined country code normalization logic across sign-up rules and user
profile pages for consistency.
* **Documentation**
* Clarified country code field messaging from "ISO code" to "2-letter
country code" terminology for better user guidance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Enhances sign-up process with Turnstile integration for fraud
protection. Builds on top of fraud-protection-temp-emails.
Made with [Cursor](https://cursor.com)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Cloudflare Turnstile bot-protection across signup/sign-in flows
(including SDK JSON mode).
* Email deliverability checks via Emailable.
* Sign-up risk scoring with persisted risk metrics and country code
tracking.
* UI: country-code selector, risk-score editing in user details, users
list refresh button, and Turnstile signup demo pages.
* **Bug Fixes**
* Use actual sign-up timestamp for reporting/metrics.
* **Documentation**
* Expanded knowledge base on Turnstile, risk scoring, and env
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
Co-authored-by: BilalG1 <bg2002@gmail.com>
Co-authored-by: Armaan Jain <84474476+Developing-Gamer@users.noreply.github.com>
Co-authored-by: nams1570 <amanganapathy@gmail.com>
This PR implements unified AI endpoint and custom dashboards.
**Unified AI Endpoint**
We now use a single endpoint throughout the codebase that makes the call
to openrouter. Specifically, email drafts, email templates, email
themes, wysiwyg, cmd centre ai search and docs ai, all use this unified
ai endpoint. All the tools are defined in the backend, all the prompts
exist in the backend.
How to review this PR for unified ai endpoint:
This PR will be easier to review if we look at the different folders
that were affected.
under packages - We added streaming functionality, and made renaming
changes
under docs - there are three files that have changed
package.json - we updated the package (we were previously using a very
old version of the package)
route.ts - we changed the call from a direct call to openrouter to the
unified ai endpoint
ai-chat.tsx - because of updating the package, we had to make changes to
adapt to the latest versions of the package
under backend
route.ts - the main unified ai endpoint. this endpoint uses various
support files
forward.ts - this is the forward to production functionality
models.ts - consists of the models, and the rules for selecting those
models
prompts.ts - consists of the base prompt + specific system prompts
depending upon the usage
schema.ts
every single file under ai/tools folder - which as the name suggests,
consists of the implementations of the different tools that can be
provided to the llm
route-handlers - added support for streaming to SmartRoute and response
under dashboard
ai-search/route.ts - refactored the file to use unified ai endpoint
chat-adapters.ts - refactored the file to use unified ai endpoint and
created extra checks for the ai generated code
**Custom Dashboards**
We let the user write their query in english. We then use AI to create
dashboards that are interactive, live and savable. This PR includes a
new package called dashboard-ui-components. This package has components
that are used in the dashboard and more importantly, these components
are being imported from esm in the ai generated code for custom
dashboards. We also change the bar at the top for the products pages.
How to review this PR:
Review the new package (package/dashboard-ui-components), the setup and
the files inside it.
Review the schema changes in stack-shared/src
Review the changes in dashboard. The following changes have been made
Updated the design-components folder since we moved the dashboard
components to the new package
Updated imports for these components accordingly
Updated the title bar of the product pages
Created the files for custom dashboards under the dashboards folder and
components under commands/create-dashboard
Created a script under dashboard/scripts that generates the file with
type definitions that would go to the llm
Review the backend
Started using unified ai endpoint
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added custom dashboards feature allowing users to create and manage
personalized dashboards with AI assistance.
* Integrated AI-assisted dashboard code generation with visual preview
and editing capabilities.
* Introduced new AI query endpoints supporting stream and generate modes
with configurable model quality/speed settings.
* **Improvements**
* Reorganized UI components into a dedicated component library package
for better code reuse.
* Enhanced chat architecture with improved message handling and tool
integration.
* Updated AI provider integration with improved configuration
management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>
Co-authored-by: Bilal Godil <bg2002@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
### Context
We didn't have an easy place for a user to see their domain statistics
and track their sent emails, either overall or by draft. Additionally,
there was scope creep with the sidebar, where we were supporting more
pages. Our emails landing page was also rather confusing, especially
toggling/ working with different email server types. So, we decide to
add a "sent" page, to track email logs and email statistics, as well as
let users temporarily override their sending limits if need be.
Additionally, a user may want to see a particular email in more detail:
what stage is it in? How did it proceed through time? How can I pause
the sending of this email or change the scheduled time or edit the code?
We allow for that to happen.
### Summary of Changes
#### New Pages
1. **Sent Page:** A Domain Reputation card lets you track how many of
your sent emails were bounced or marked as spam as well as how much
capacity you have left. We also provide a temporary override, where you
can use up to 4 times your capacity for a limited period of time.
Additionally, we provide an email log that lets you see the recently
sent emails. You can also toggle this view from a "list all emails" to
"group by template/draft" which shows stats for each template/draft id
(i.e a bar showing how many emails were sent, are pending, were marked
as spam, were bounced etc, and the total number of emails sent with that
template or draft). Clicking on an email in the list all view takes you
to the "email-viewer" endpoint for that email (see below). Clicking on a
template/draft in the group by view takes you to a page where you can
see the statistics for that template/draft in more detail (the "send"
stage view for that template/draft, as referenced below).
2. **Settings Page:** This is a new page we created because the old
"emails" landing page wasn't doing its job. This page is to track all
the email settings. Currently, we put in 2 sections. A "theme settings"
card where users can see their active theme and click on a button to be
navigated to the themes page. This is necessary as we remove themes from
the sidebar. The other section is a card for email server and domain
configuration - you can change your server type and adjust the settings
or send a test email. It's cleaner and less noisy.
3. **Drafts Page**: There are a lot of changes here. On the landing
page, we actually separate out the drafts into "active drafts" and
"draft history" because drafts are meant to be fire-and-forget, not
reusable. We also add the functionality to create a draft from a
template. This was tricky to manage because templates rely on template
variables which sent to the backend along with the code and injected
during render time. We deal with this by having AI rewrite the template
source code to remove any references to template variables and to make
the draft standalone. The drafts page has been separated into a
stepper-controlled multi stage process:
draft->recipients->schedule->sent. Sent is a read only view that shows
you the statistics of the emails sent using that draft, as mentioned
earlier. You can also see the sent view of a historical draft. You can
also bulk pause/cancel any unsent emails from the sent view of the
drafts.
4. **Sidebar Updates**: The email sidebar now doesn't show "themes" or
"emails" (the old landing page), but it does show "settings" and "sent",
and the default landing page for emails is "sent".
5. **Email Viewer**: When you click on an individual email, you get
navigated here. This has a timeline showing the progress of the email on
the right, and some optional info for the user that's toggleable on the
right bottom, while having either a preview of the email if it's sent or
a way to edit it. You can also change the scheduledAt date of an email
if it hasn't already been sent.
#### Bug Fixes
1. **Search in `TeamMemberSearchTable`**: This was broken. Every time
you tried to enter or remove a character, it would trigger skeleton
loading that overlapped the search bar too, preventing you from
adding/removing more. This was caused because the `useUser` hook
eventually ended up calling a `use` hook, which throws a promise that
triggers a suspense. This, coupled with the fact that the implementation
of `TeamMemberSearchTable` involved a prop-drilling/ dependency
inversion approach to passing down its toolbar to a base table
component, meant the suspense would cover the toolbar too and couldn't
be scoped to just the table. A refactor has gotten rid of the need for
those base components while fixing tables in `payments/customers`,
`teams/team_id`, and `payments/transactions` on top of the existing use
in email drafts recipients stage. We also dedupped some code.
2. **Stale draft fetches on draft landing page**: `useEmailDrafts` uses
an asyncCache to cache the fetched drafts. It is used on the drafts
landing page to render the drafts. When a draft is sent, its `sentAt` is
marked versus when it is still active, it is marked as null. The cache
was stale and so navigating to the landing page after firing off a draft
would errorneously represent that draft as still active and indeed, even
allow you to edit it and fire it again. This violated the principle of
drafts being fire and forget. This has been dealt with by adding
functionality to refresh the draft cache upon firing off a draft.
#### Other Changes
1. We bumped up the base time for the exponential send attempt retry
backoff in `email-queue-step` to 20 seconds. The previous base was two
seconds, and this effectively just made it wait until the next iteration
of the `email-queue-step` cron job or at most an iteration that wasn't
too far away. When an outage with our provider happens, it may take a
while for it to be resolved, so a longer backoff is justified
2. We transitioned the themes page and the templates page to using the
new components, though deeper UI refactors for them were out of scope
for this ticket.
3. We implement a "temporarily increase capacity" button, that bumps up
the throughput/ capacity limit fourfold for a user for a given period of
time. It works like this:
> Clicking the button sets a boost expiredat time.
> When this time is set and still valid, the capacity rate is multiplied
by 4.
> When the button is clicked, trigger a loading spinner until the route
finishes processing.
> When the timer runs out, we reset the button back to its original
state.
> We dont need to wrap the onclick with runAsyncWithAlert because the
component does that already.
4. We add a new default theme: a colorful theme with a lavender base.
This was mainly done so we could have three times in a theme showcase in
the settings page.
### UI Demos
**Sent Page Demo:**
https://github.com/user-attachments/assets/19294a90-bb65-4f00-9a97-111f6c08287f
**Drafts Page Demo**
https://github.com/user-attachments/assets/847609ef-d699-470c-a699-297bb9e17f04
**Settings Page Demo**
https://github.com/user-attachments/assets/190a3829-036a-4f57-89c0-a873bef5a7ce
**Email Viewer Page Demo**
https://github.com/user-attachments/assets/3bc50159-4acb-4865-a4dd-830c84ee4235
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a new "Hosted Components" app with its own app shell, routing,
auth-aware UI, a handler route, and a welcome page showing the signed-in
user.
* **Chores**
* Added dev tooling and configuration for the new app (build, lint,
typecheck, Vite/TS, package manifest) and updated dev env API URL.
* **Tests**
* Excluded the new app from the test workspace.
* **Bug Fixes**
* Suppressed noisy console errors for a specific internal sentinel and
clarified related error messaging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Managed email domain onboarding: setup, DNS provisioning,
verification, status checks, and apply flow (Resend-backed).
* **UI**
* Project email settings: managed-provider setup dialog, managed sender
fields, status display, and test-send mapping.
* **Integrations**
* DNS provider automation and Resend webhook handling for domain status
updates; scoped keys for sending.
* **API**
* Admin endpoints / client APIs to setup, check, list, and apply managed
email domains.
* **Tests**
* End-to-end tests covering the full onboarding flow.
* **Chores**
* Added environment variables and config schema support for Resend and
DNS integrations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
### Context
We're looking at implementing plan pricing. While doing so, we
encountered a problem with Stripe.
**Problem:** when we run a stripe operation (purchase), the product info
is encoded as part of the stripe metadata request. Stripe encodes
metadata as key-value pairs, and the [value has a limit of 500
chars](https://docs.stripe.com/metadata#data). We do this because once
we run the stripe operation, stripe fires a webhook event which is
caught by our stripe webhook handler syncStripeSubscriptions. This gets
the stripe metadata info from the event and then updates our db in
prisma.
### Summary of Changes
We add a `ProductVersion` table and only pass the `productVersionId` via
stripe metadata instead of the whole product json. This
`productVersionId` is created by hashing the `productJson`. Since the
same product may be ordered differently without being intrinsically
different, we add a helper function for ensuring a canonical order to
the json. We also pass tenancy id and product id to the table.
Since there are existing subscriptions which used to pass the
productJson via metadata, we ensure backwards compatibility.
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **High Risk**
> Touches authentication and OAuth token/authorize flows and changes how
client requests are validated, so regressions could cause widespread
login/client-access failures. Also includes a data migration that alters
effective security posture for existing projects.
>
> **Overview**
> Adds a **project-level toggle**
(`project.requirePublishableClientKey`) to control whether client
requests/OAuth flows must include a publishable client key, including a
DB migration that backfills existing projects to require it.
>
> Backend auth now treats the publishable client key as *optional when
allowed*, introducing a public sentinel (`__stack_public_client__`) and
returning a new specific error
(`PUBLISHABLE_CLIENT_KEY_REQUIRED_FOR_PROJECT`) across smart request
auth + OAuth `authorize`/`callback`/`token` endpoints.
>
> Dashboard and SDKs update key generation/display and request
construction to handle missing publishable keys, expose an advanced
toggle on the Project Keys page, and extend internal config overrides to
support a new `project` level; E2E/tests and schema fuzzing are expanded
accordingly, and CI adds a forward-compat migration check job when
back-compat fails.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
5d06c08613. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Project-level config to require publishable client keys; migration
applied to existing projects.
* **Improvements**
* Auth flows now support optional publishable client keys with explicit
validation and a sentinel for keyless OAuth.
* Dashboard/UI and SDKs handle publishable keys as optional and
conditionally show/generate them.
* Admin/client APIs extended to manage project-level overrides.
* **Bug Fixes**
* Key validation behavior aligned with project config.
* **Tests**
* Expanded E2E and unit tests covering optional/required publishable-key
scenarios.
* **Documentation**
* Spec and knowledge docs updated to describe the sentinel and config
behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
