mirror of
https://github.com/stack-auth/stack.git
synced 2026-06-13 21:01:21 +08:00
### Suggested Review Areas Please see `plans.ts` and `seed.ts` to verify whether the item caps are where they should be. Outside of that, each commit should be atomic so stepping through the commits should give you an idea of how I implemented each limit. ### Discussion Something to discuss: when a user cancels team/growth we regrant free fine, but any extra-seats they had just keeps billing. So they end up paying ~$29/mo per extra-seat on top of free's 1 seat, which is strictly worse than just staying on team. This surfaced while manually testing this PR, we only enforce the add-on base requirement at purchase time, nothing cascades on cancel. Should we cascade cancel add ons? ### Context Now that we have a stable suite of products for stack-auth, we want to limit the items under each product a customer has access to based on their plan. So for example, a free plan user has a certain amount of emails they can send out each month, and so on. We try to implement limits in this PR. ### Summary of Changes Implemented hard limits for dashboard admins, analytics per-query timeouts, sent email monthly capacity, events, and session replays. Implemented a soft cap for auth users (where if there's a signup beyond the limit, we log it to sentry so we can manually choose to email that user/team). For auth users, we do not block new user sign ups once plan limit has been hit. We also don't degrade or impact the customer experience. It logs to sentry and it is up to us to take manual action to email the user to upgrade the plan. Also, implementation wise, we count all the users across all the projects for this team and compare it to their plan item limit, rather than debiting items like we do for other approaches. As a soft cap, this should be fine plus this is a better source of truth. For email capacity, we operate a monthly limit of emails. Once this is hit, no more emails can be sent until the next month/ a plan upgrade. These emails will be treated as a send error, so they can be manually resent once the capacity is reset. With respect to the `email-queue` state engine, they go from `SENDING`->`SERVER_ERROR`, hooking into the existing state engine flow, with an external error that shows it's because of the rate limit. This is cleaner than inventing a new state that is identical for all intents and purposes to `SERVER_ERROR`. We check in processSingleEmail since that maps to the sending state. For analytics query timeouts, the backend route accepts a timeout parameter with the request. The way we implement the timeout for each query is by taking the `min(request_timeout,plan_timeout)` and using that. This determines how long a query can run for. For analytics events, there are server-side events (like refresh token refreshes or sign up rule triggers) and client side events (like page views or clicks). When these events occur, they are written to the events table in clickhouse. We choose to implement a hard cap for the total events, not just server side or client side. Once the cap is hit, we stop storing the events and display a banner on the analytics page. A different banner renders when we are at >=80% of total plan capacity. For session replays, we stop creating new session replays when the limit is hit. Old replays can still have chunks appended to them. The source of truth here is the session replay table- a new replay corresponds to a new row in the table. We have similar banners as to the events. Dashboard admins should be 4 for both team and unlimited. #### Implementation Caveats For debiting items across these limits, we now use `tryDecreaseQuantity` at the beginning. This means we debit first if possible before conducting the action (like writing events to clickhouse). In practice, this means that if clickhouse fails, then the user is debited for something that doesn't happen. However trying to build a refund workaround would be very clunky, and also, clickhouse is reliable. For debits that are very small in the order of things (say, 200 items on a 100k plan), it doesn't mean much. For emails, we don't debit items if it's a retry. This prevents the user for being charged multiple times for effectively one email. ### UI Changes The only UI changes in this PR are having certain banners render in analytics when a customer is approaching/ is at their monthly limit of session replays or events. ### Out of Scope for this PR We do not have metered pricing yet, so events/session replays/ email use beyond the limits cannot be charged yet. This is why for this implementation, we rely on hard and soft caps. We do not implement payment per-transaction pricing yet. That is deferred to a followup PR. The UI for the onboarding call will be set up as part of the overall onboarding flow which doesn't exist yet, so it has been deferred. Since the UI for the dashboard home page and project/account settings is currently being reworked, finding a better spot for plan upgrades is not handled in this PR. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Session replays added as a monthly included entitlement; onboarding calls added to Team/Growth plans. Dashboard banners warn about analytics-event and session-replay limits. Projects page adds extra-seat flow and improved invitation error handling. * **Behavior Changes** * Monthly renewal semantics for emails-per-month and analytics-events; analytics query timeouts now respect plan limits and are clamped. Email sends, analytics events, and new session creation are blocked when quotas are exhausted. Growth plan seats set to 4. * **Tests** * E2E and unit tests added to verify quota enforcement and free-plan regranting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Mantra <87142457+mantrakp04@users.noreply.github.com>
289 lines
8.5 KiB
TypeScript
289 lines
8.5 KiB
TypeScript
import { KnownErrors } from "@stackframe/stack-shared";
|
|
import { DEFAULT_EMAIL_THEME_ID, DEFAULT_TEMPLATE_IDS } from "@stackframe/stack-shared/dist/helpers/emails";
|
|
import { wait } from "@stackframe/stack-shared/dist/utils/promises";
|
|
import { it } from "../helpers";
|
|
import { withPortPrefix } from "../helpers/ports";
|
|
import { createApp } from "./js-helpers";
|
|
|
|
async function setupEmailServer(adminApp: any) {
|
|
const project = await adminApp.getProject();
|
|
await project.updateConfig({
|
|
emails: {
|
|
server: {
|
|
isShared: false,
|
|
host: "localhost",
|
|
port: Number(withPortPrefix("29")),
|
|
username: "test",
|
|
password: "test",
|
|
senderEmail: "test@example.com",
|
|
senderName: "Test User",
|
|
},
|
|
},
|
|
});
|
|
}
|
|
|
|
it("should successfully send email with HTML content", async ({ expect }) => {
|
|
const { adminApp, serverApp } = await createApp();
|
|
await setupEmailServer(adminApp);
|
|
|
|
const user = await serverApp.createUser({
|
|
primaryEmail: "test@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
await expect(serverApp.sendEmail({
|
|
userIds: [user.id],
|
|
html: "<h1>Test Email</h1><p>This is a test email with HTML content.</p>",
|
|
subject: "Test Subject",
|
|
})).resolves.not.toThrow();
|
|
});
|
|
|
|
it("should successfully send email with template", async ({ expect }) => {
|
|
const { adminApp, serverApp } = await createApp();
|
|
await setupEmailServer(adminApp);
|
|
|
|
const user = await serverApp.createUser({
|
|
primaryEmail: "test@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
await expect(serverApp.sendEmail({
|
|
userIds: [user.id],
|
|
templateId: DEFAULT_TEMPLATE_IDS.sign_in_invitation,
|
|
variables: {
|
|
teamDisplayName: "Test Team",
|
|
signInInvitationLink: "https://example.com",
|
|
},
|
|
subject: "Welcome!",
|
|
})).resolves.not.toThrow();
|
|
});
|
|
|
|
it("should successfully send email to multiple users", async ({ expect }) => {
|
|
const { adminApp, serverApp } = await createApp();
|
|
await setupEmailServer(adminApp);
|
|
|
|
const user1 = await serverApp.createUser({
|
|
primaryEmail: "test1@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
const user2 = await serverApp.createUser({
|
|
primaryEmail: "test2@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
await expect(serverApp.sendEmail({
|
|
userIds: [user1.id, user2.id],
|
|
html: "<p>Bulk email test</p>",
|
|
subject: "Bulk Email Test",
|
|
})).resolves.not.toThrow();
|
|
});
|
|
|
|
it("should send email with theme customization", async ({ expect }) => {
|
|
const { adminApp, serverApp } = await createApp();
|
|
await setupEmailServer(adminApp);
|
|
|
|
const user = await serverApp.createUser({
|
|
primaryEmail: "test@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
await expect(serverApp.sendEmail({
|
|
userIds: [user.id],
|
|
html: "<p>Themed email test</p>",
|
|
subject: "Themed Email",
|
|
themeId: DEFAULT_EMAIL_THEME_ID,
|
|
})).resolves.not.toThrow();
|
|
});
|
|
|
|
it("should send email with notification category", async ({ expect }) => {
|
|
const { adminApp, serverApp } = await createApp();
|
|
await setupEmailServer(adminApp);
|
|
|
|
const user = await serverApp.createUser({
|
|
primaryEmail: "test@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
await expect(serverApp.sendEmail({
|
|
userIds: [user.id],
|
|
html: "<p>Notification email test</p>",
|
|
subject: "Notification Email",
|
|
notificationCategoryName: "Transactional",
|
|
})).resolves.not.toThrow();
|
|
});
|
|
|
|
it("should throw RequiresCustomEmailServer error when email server is not configured", async ({ expect }) => {
|
|
const { serverApp } = await createApp();
|
|
|
|
const user = await serverApp.createUser({
|
|
primaryEmail: "test@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
await expect(serverApp.sendEmail({
|
|
userIds: [user.id],
|
|
html: "<p>This should fail</p>",
|
|
subject: "Test Email",
|
|
})).rejects.toThrow(KnownErrors.RequiresCustomEmailServer);
|
|
});
|
|
|
|
it("should handle non-existent user IDs", async ({ expect }) => {
|
|
const { adminApp, serverApp } = await createApp();
|
|
await setupEmailServer(adminApp);
|
|
|
|
// Use a properly formatted UUID that doesn't exist
|
|
await expect(serverApp.sendEmail({
|
|
userIds: ["123e4567-e89b-12d3-a456-426614174000"],
|
|
html: "<p>Non-existent user test</p>",
|
|
subject: "Test Email",
|
|
})).rejects.toThrow(KnownErrors.UserIdDoesNotExist);
|
|
});
|
|
|
|
it("should handle missing required email content", async ({ expect }) => {
|
|
const { adminApp, serverApp } = await createApp();
|
|
await setupEmailServer(adminApp);
|
|
|
|
const user = await serverApp.createUser({
|
|
primaryEmail: "test@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
await expect(serverApp.sendEmail({
|
|
userIds: [user.id],
|
|
subject: "Test Email",
|
|
} as any)).rejects.toThrow(KnownErrors.SchemaError);
|
|
});
|
|
|
|
it("should handle html and templateId at the same time", async ({ expect }) => {
|
|
const { adminApp, serverApp } = await createApp();
|
|
await setupEmailServer(adminApp);
|
|
|
|
const user = await serverApp.createUser({
|
|
primaryEmail: "test@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
await expect(serverApp.sendEmail({
|
|
userIds: [user.id],
|
|
html: "<p>Test Email</p>",
|
|
templateId: DEFAULT_TEMPLATE_IDS.sign_in_invitation,
|
|
subject: "Test Email",
|
|
} as any)).rejects.toThrow(KnownErrors.SchemaError);
|
|
});
|
|
|
|
it("should provide delivery statistics", async ({ expect }) => {
|
|
const { adminApp, serverApp } = await createApp();
|
|
await setupEmailServer(adminApp);
|
|
|
|
const user = await serverApp.createUser({
|
|
primaryEmail: "stats@example.com",
|
|
primaryEmailVerified: true,
|
|
});
|
|
|
|
// Give Bulldozer's pg_cron tick time to materialise the billing team's
|
|
// `emails_per_month` quota from its freshly-granted free plan. Without
|
|
// this wait the first email gets quota-blocked into a permanent
|
|
// server-error terminal and `stats.hour.sent` never reaches 1.
|
|
await wait(2000);
|
|
|
|
await serverApp.sendEmail({
|
|
userIds: [user.id],
|
|
html: "<p>Stats</p>",
|
|
subject: "Stats",
|
|
});
|
|
|
|
let info;
|
|
for (let i = 0; ; i++) {
|
|
info = await serverApp.getEmailDeliveryStats();
|
|
if (info.stats.hour.sent >= 1) break;
|
|
if (i >= 50) {
|
|
throw new Error(`Timed out waiting for email delivery stats to reflect sent email: ${JSON.stringify(info)}`);
|
|
}
|
|
await wait(500);
|
|
}
|
|
|
|
expect(info).toMatchInlineSnapshot(`
|
|
{
|
|
"capacity": {
|
|
"boost_expires_at": null,
|
|
"boost_multiplier": 1,
|
|
"is_boost_active": false,
|
|
"penalty_factor": 1,
|
|
"rate_per_second": 27.777779320987655,
|
|
},
|
|
"stats": {
|
|
"day": {
|
|
"bounced": 0,
|
|
"marked_as_spam": 0,
|
|
"sent": 1,
|
|
},
|
|
"hour": {
|
|
"bounced": 0,
|
|
"marked_as_spam": 0,
|
|
"sent": 1,
|
|
},
|
|
"month": {
|
|
"bounced": 0,
|
|
"marked_as_spam": 0,
|
|
"sent": 1,
|
|
},
|
|
"week": {
|
|
"bounced": 0,
|
|
"marked_as_spam": 0,
|
|
"sent": 1,
|
|
},
|
|
},
|
|
}
|
|
`);
|
|
});
|
|
|
|
it("should send test email with custom SMTP configuration", async ({ expect }) => {
|
|
const { adminApp } = await createApp();
|
|
|
|
// First configure the email server
|
|
await setupEmailServer(adminApp);
|
|
|
|
// Get the project to access the email config
|
|
const project = await adminApp.getProject();
|
|
const config = await project.getConfig();
|
|
|
|
// Verify config is not shared
|
|
expect(config.emails.server.isShared).toBe(false);
|
|
|
|
// Give Bulldozer's pg_cron tick time to materialise the billing team's
|
|
// `emails_per_month` quota from its freshly-granted free plan; otherwise
|
|
// the `tryDecreaseQuantity` inside the route rejects with
|
|
// ItemQuantityInsufficientAmount before SMTP is ever dialled.
|
|
await wait(2000);
|
|
|
|
// Send a test email
|
|
const result = await adminApp.sendTestEmail({
|
|
recipientEmail: "test-recipient@example.com",
|
|
emailConfig: {
|
|
host: config.emails.server.host!,
|
|
port: config.emails.server.port!,
|
|
username: config.emails.server.username!,
|
|
password: config.emails.server.password!,
|
|
senderEmail: config.emails.server.senderEmail!,
|
|
senderName: config.emails.server.senderName!,
|
|
}
|
|
});
|
|
|
|
expect(result.status).toBe('ok');
|
|
});
|
|
|
|
it("should fail to send test email with shared server configuration", async ({ expect }) => {
|
|
const { adminApp } = await createApp();
|
|
|
|
// Don't configure custom email server, so it defaults to shared
|
|
const project = await adminApp.getProject();
|
|
const config = await project.getConfig();
|
|
|
|
// Verify config is shared
|
|
expect(config.emails.server.isShared).toBe(true);
|
|
|
|
// Attempting to send test email with shared config should fail in the UI
|
|
// (This test documents the expected behavior in the dashboard UI)
|
|
});
|