### Object of this PR
This PR is NOT a monolithic series of fixes for the payments suite + a
complete rework. Its aims were
a) introducing and robustly testing the bulldozer db system
b) reworking the payments underlying architecture to use bulldozer for
correctness and scalability
c) Achieving parity with the old payments system excepting a few changes
like ensuring correctness of the ledger algo
There may still be some work to do with handling refunds, decoupling the
concepts of purchases from that of products, and some other things.
### Ledger Algorithm
This has been tuned and fixed. Item removals i.e negative item quantity
changes will apply to the soonest expiring item grant i.e positive item
quantity change. This is what is best for the user. Item grants can also
expire, and when they expire we obviate whatever is left of their
original capacity (meaning after all the removals that were applied to
it). Our ledger algo is applied via Bulldozer, so automatic
re-computation is handled when a new grant/ removal is inserted in the
middle of the existing ones.
### Things we got rid of
* No more automatic support for default products. You can use $0 plan
provisions to accomplish the same effect but it's manual
* Negative item quantity changes (i.e item removals) no longer can have
expiries
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced payment processing pipeline with improved data consistency
and state management.
* Advanced refund handling with comprehensive transaction tracking.
* Better tracking and management of customer item quantities and owned
products.
* Improved subscription lifecycle management including period-end
handling.
* **Bug Fixes**
* Fixed payment data integrity verification.
* Improved handling of edge cases in refund scenarios.
* **Chores**
* Updated cSpell configuration with additional words.
* Expanded developer documentation for linting workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
Co-authored-by: Aadesh Kheria <kheriaaadesh@gmail.com>
Co-authored-by: Mantra <87142457+mantrakp04@users.noreply.github.com>
DB migration compat / Back-compat — Current branch migrations with ${{ needs.check-migrations-changed.outputs.base_branch }} branch code (push) Has been cancelled
DB migration compat / Forward-compat — Current branch code with ${{ needs.check-migrations-changed.outputs.base_branch }} branch migrations (push) Has been cancelled
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a new "Hosted Components" app with its own app shell, routing,
auth-aware UI, a handler route, and a welcome page showing the signed-in
user.
* **Chores**
* Added dev tooling and configuration for the new app (build, lint,
typecheck, Vite/TS, package manifest) and updated dev env API URL.
* **Tests**
* Excluded the new app from the test workspace.
* **Bug Fixes**
* Suppressed noisy console errors for a specific internal sentinel and
clarified related error messaging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>
<!--
ONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- RECURSEML_SUMMARY:START -->
## High-level PR Summary
This PR changes the default development ports for several background
services to avoid conflicts. PostgreSQL moves from port `5432` to
`8128`, Inbucket SMTP from `2500` to `8129`, Inbucket POP3 from `1100`
to `8130`, and the OpenTelemetry collector from `4318` to `8131`. All
references across configuration files, Docker Compose setups,
environment files, CI/CD workflows, test files, and documentation have
been updated to reflect these new port assignments. A knowledge base
document has been added to document the new port mappings.
⏱️ Estimated Review Time: 15-30 minutes
<details>
<summary>💡 Review Order Suggestion</summary>
| Order | File Path |
| --- | --- |
| 1 | `claude/CLAUDE-KNOWLEDGE.md` |
| 2 | `apps/dev-launchpad/public/index.html` |
| 3 | `docker/dependencies/docker.compose.yaml` |
| 4 | `docker/emulator/docker.compose.yaml` |
| 5 | `apps/backend/.env` |
| 6 | `apps/backend/.env.development` |
| 7 | `docker/server/.env.example` |
| 8 | `package.json` |
| 9 | `.devcontainer/devcontainer.json` |
| 10 | `apps/e2e/.env.development` |
| 11 | `.github/workflows/check-prisma-migrations.yaml` |
| 12 | `.github/workflows/docker-server-test.yaml` |
| 13 | `.github/workflows/e2e-api-tests.yaml` |
| 14 | `.github/workflows/e2e-source-of-truth-api-tests.yaml` |
| 15 | `.github/workflows/restart-dev-and-test.yaml` |
| 16 |
`apps/e2e/tests/backend/endpoints/api/v1/internal/email-drafts.test.ts`
|
| 17 | `apps/e2e/tests/backend/endpoints/api/v1/internal/email.test.ts`
|
| 18 | `apps/e2e/tests/backend/endpoints/api/v1/send-email.test.ts` |
| 19 |
`apps/e2e/tests/backend/endpoints/api/v1/unsubscribe-link.test.ts` |
| 20 | `apps/e2e/tests/backend/workflows.test.ts` |
| 21 | `docs/templates/others/self-host.mdx` |
</details>
[](https://discord.gg/n3SsVDAW6U)
[
<!-- RECURSEML_SUMMARY:END -->
<!-- ELLIPSIS_HIDDEN -->
----
> [!IMPORTANT]
> This PR introduces customizable development ports using
`NEXT_PUBLIC_STACK_PORT_PREFIX`, updating configurations, documentation,
and tests accordingly.
>
> - **Behavior**:
> - Default development ports for services are now customizable via
`NEXT_PUBLIC_STACK_PORT_PREFIX`.
> - PostgreSQL port changed from `5432` to
`${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}28`.
> - Inbucket SMTP port changed from `2500` to
`${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}29`.
> - Inbucket POP3 port changed from `1100` to
`${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}30`.
> - OpenTelemetry collector port changed from `4318` to
`${NEXT_PUBLIC_STACK_PORT_PREFIX:-81}31`.
> - **Configuration**:
> - Updated `docker.compose.yaml` to use new port variables for services
like PostgreSQL, Inbucket, and OpenTelemetry.
> - Environment files in `apps/backend`, `apps/dashboard`, and
`apps/e2e` updated to use `NEXT_PUBLIC_STACK_PORT_PREFIX`.
> - `package.json` scripts updated to reflect new port configurations.
> - **Documentation**:
> - Added `CLAUDE-KNOWLEDGE.md` to document new port mappings.
> - Updated `self-host.mdx` to reflect new port configurations.
> - **Testing**:
> - Updated test files in `apps/e2e/tests` to use new port
configurations.
> - Added `helpers/ports.ts` for port-related utilities in tests.
>
> <sup>This description was created by </sup>[<img alt="Ellipsis"
src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=stack-auth%2Fstack-auth&utm_source=github&utm_medium=referral)<sup>
for 76ef55f58f. You can
[customize](https://app.ellipsis.dev/stack-auth/settings/summaries) this
summary. It will automatically update as commits are pushed.</sup>
----
<!-- ELLIPSIS_HIDDEN -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enable configurable development ports via a
NEXT_PUBLIC_STACK_PORT_PREFIX, allowing parallel local environments with
custom port prefixes.
- **Bug Fixes**
- Updated local service port mappings and CI/workflow settings so
tooling and tests use the new prefixed ports consistently.
- **Documentation**
- Added docs and contributor guidance for running multiple parallel
workspaces with custom port prefixes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: N2D4 <N2D4@users.noreply.github.com>
<!--
Make sure you've read the CONTRIBUTING.md guidelines:
https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md
-->
<!-- ELLIPSIS_HIDDEN -->
----
> [!IMPORTANT]
> Enhances email sending API with template support, improved error
handling, and new interfaces, while adding comprehensive tests and
updating rendering logic.
>
> - **Behavior**:
> - Adds support for sending emails using templates with variables and
optional theming in `send-email/route.tsx`.
> - Introduces per-user notification category checks before sending
emails.
> - Adds optional unsubscribe link in email themes.
> - **Error Handling**:
> - Refines error handling in `send-email/route.tsx` for missing
content, non-existent user IDs, and shared email server configurations.
> - Uses `KnownErrors` for specific error cases.
> - **API Changes**:
> - Adds new interfaces and methods for email sending in
`server-interface.ts` and `admin-interface.ts`.
> - Removes deprecated email sending methods from admin interfaces.
> - **Testing**:
> - Adds e2e tests in `email.test.ts` for various email sending
scenarios, including HTML content, templates, and error cases.
> - **Misc**:
> - Updates email rendering logic in `email-rendering.tsx` to handle new
template and theme options.
> - Simplifies import statements and cleans up code structure across
multiple files.
>
> <sup>This description was created by </sup>[<img alt="Ellipsis"
src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=stack-auth%2Fstack-auth&utm_source=github&utm_medium=referral)<sup>
for 1d5a056699. You can
[customize](https://app.ellipsis.dev/stack-auth/settings/summaries) this
summary. It will automatically update as commits are pushed.</sup>
----
<!-- ELLIPSIS_HIDDEN -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced email sending to support both raw HTML and template-based
emails with variables and optional theming.
* Added per-user notification category checks before sending emails.
* Email themes now support an optional unsubscribe link in the footer.
* **Improvements**
* Updated email rendering to pass the unsubscribe link as a prop to
themes.
* Refined error handling for email sending.
* Improved flexibility of email sending options and result reporting.
* **API Changes**
* Introduced new interfaces and methods for sending emails on the server
side, including detailed result reporting.
* Removed deprecated admin-side email sending methods and interfaces.
* Added new types for email sending options and results.
* **Bug Fixes**
* Fixed navigation and property naming inconsistencies in dashboard
email template editing and sending flows.
* **Chores**
* Simplified import statements and cleaned up internal code structure.
* Updated Docker environment for freestyle mock service to use Bun
runtime and adjusted port mappings.
* **Tests**
* Added comprehensive tests covering email sending scenarios, including
error handling and multi-user support.
* Updated existing tests to reflect refined email subjects, template
rendering, and unsubscribe link features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>
Note for deployment: This PR needs to add some env vars
<!-- ELLIPSIS_HIDDEN -->
----
> [!IMPORTANT]
> Adds S3-compatible storage for profile images with S3Mock for local
development, updating environment variables, Docker configurations, and
tests.
>
> - **Behavior**:
> - Adds S3-compatible storage for user, team, and team member profile
images using `uploadAndGetUrl()` in `s3.tsx`.
> - Integrates S3Mock for local development in `docker.compose.yaml` and
`emulator/docker.compose.yaml`.
> - Updates `crud.tsx` files for `users`, `teams`, and
`team-member-profiles` to use S3 for profile images.
> - **Environment**:
> - Adds S3-related environment variables to `.env.development`.
> - Updates `package.json` to include `@aws-sdk/client-s3`.
> - **Testing**:
> - Modifies test cases in `teams.test.ts` and `users.test.ts` to
validate S3 URL behavior.
> - Updates `auto-migration.tests.ts` for concurrent migration handling.
> - **Documentation**:
> - Updates `self-host.mdx` to include S3 storage requirements.
>
> <sup>This description was created by </sup>[<img alt="Ellipsis"
src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=stack-auth%2Fstack-auth&utm_source=github&utm_medium=referral)<sup>
for 901f0b56cd. You can
[customize](https://app.ellipsis.dev/stack-auth/settings/summaries) this
summary. It will automatically update as commits are pushed.</sup>
----
<!-- ELLIPSIS_HIDDEN -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for uploading and storing user, team, and team member
profile images using S3-compatible storage.
* Integrated S3Mock for local development and testing of storage
features.
* Added new services to Docker Compose configurations for S3Mock and
related dependencies.
* Introduced image validation and processing for base64-encoded images
before upload.
* **Bug Fixes**
* Profile image upload now returns a storage URL instead of echoing back
the base64 data.
* **Documentation**
* Updated self-hosting documentation to include S3 storage requirements
and usage.
* **Chores**
* Added and updated environment variables for S3 storage configuration.
* Updated backend dependencies to include the AWS S3 SDK.
* Enhanced test cases to validate new image upload and URL behaviors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>
