mirror of
https://github.com/stack-auth/stack.git
synced 2026-07-20 21:29:36 +08:00
## Problem
Session replay batches are uploaded as **raw, uncompressed JSON**. A
single large rrweb full snapshot (>900 KB) is dropped client-side in
`SessionRecorder._flush`, and any batch near the server's 1 MB body
limit is rejected. Meanwhile the **analytics events** path right next
door already gzips on the wire (`encodeAnalyticsBody` + a server-side
gunzip with zip-bomb guards) — replays just never got the same
treatment.
Notably, `dev` was already **red** on the SDK test `sends a single
oversized event alone without dropping it` (the code dropped at 900 KB
while the test expected the event to be sent). This change makes that
test pass.
## What this does
Ports the proven analytics compression pattern to session replays:
- **`packages/shared` (client transport):** generalize
`encodeAnalyticsBody` → `encodeGzipJsonBody` and wire it into
`sendSessionReplayBatch`. Replays now gzip via native
`CompressionStream` (`application/octet-stream`), with the same
fallbacks: keepalive flushes send plain JSON (so the request survives
page teardown), and a missing/throwing `CompressionStream` (Safari <
16.4) degrades to plain JSON.
- **`packages/template` (SDK recorder):** replace the 900 KB hard-drop
with `MAX_SINGLE_EVENT_BYTES = 8 MB`, kept in sync with the server's
decompressed cap. The wire is gzipped downstream, so an oversized single
event is now sent compressed instead of discarded; only an event that
could never fit *decompressed* is dropped.
- **`apps/backend` (route):** add `maybeDecodeBinaryBody` (gunzip with a
1 MB compressed cap + 8 MB `maxOutputLength` zip-bomb guard) via a
`.transform` on the batch route body schema. Plain-JSON bodies pass
through untouched.
## Why gzip / CompressionStream
Benchmarked on a real rrweb capture (Puppeteer, rrweb 1.1.3): a **1.09
MiB full snapshot → 134 KiB gzip (~8×) in ~6.6 ms**. Native
`CompressionStream` adds **0 bundle bytes** and was faster than fflate.
This mirrors what PostHog (gzip, native `CompressionStream` + fflate
fallback) and Sentry (fflate deflate in a worker) both ship — both rely
on compression, not splitting, as the primary mechanism for large
events.
## Tests
- **Client SDK** (`client-interface.test.ts`): gzip+octet-stream on
non-keepalive (round-trip + size assert), plain JSON on keepalive, and
both `CompressionStream`-unavailable fallbacks.
- **SDK recorder** (`session-replay.test.ts`): existing oversized-event
test now passes; new test that a >8 MB event is dropped while a
following normal event still sends.
- **Backend e2e** (`session-replays.test.ts`): gzipped happy path
(`compressed < 1MB < raw`), invalid gzip → 400, oversized compressed →
413, zip-bomb → 400. Verified passing against a live local backend.
## Verification notes
- `@hexclave/shared` + `@hexclave/template` typecheck pass; touched
files lint clean; SDK unit tests pass.
- Two **pre-existing, unrelated** local failures (each confirmed by
stashing this change and re-running on clean `dev`): a backend typecheck
error in `oauth/ssrf-protection.test.ts` (`NODE_ENV` read-only), and 3
session-replay **quota** e2e tests failing in the payments
`setSessionReplayItemQuantity` setup helper.
## Follow-up (out of scope)
The keepalive/unload path still sends uncompressed (64 KB browser
keepalive cap), so a very large final batch on page exit can still be
lost — not addressed here.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Gzip session replay batches so large rrweb snapshots are sent instead of
dropped. Keepalive flushes cap single-event size to avoid 413s on page
unload, and limits now include a 1 KiB envelope margin to match server
caps.
- **Bug Fixes**
- Client (`@hexclave/shared`): generalized `encodeAnalyticsBody` →
`encodeGzipJsonBody`; gzip via native `CompressionStream` for
`/session-replays/batch`; keepalive and missing/throwing
`CompressionStream` fall back to JSON.
- SDK recorder (`@hexclave/template`): normal flushes allow a single
event up to `MAX_SINGLE_EVENT_BYTES = 8 MiB - 1 KiB`; keepalive flushes
cap single-event size at `MAX_BATCH_UNCOMPRESSED_BYTES = 900 KB`; send
oversized events alone and rely on gzip; drop only above those
thresholds; log a distinct 413 warning with the count of buffered events
dropped and stop the loop.
- Backend: gunzip `application/octet-stream` bodies with caps (`1 MB`
compressed, `8 MiB` decompressed); invalid gzip → 400; oversized
compressed → 413; plain JSON passes through.
- Tests: added client encoding tests for session replays; SDK tests for
>8 MiB drop, keepalive drop, and 413 warning; backend e2e for gzip,
invalid gzip, oversized compressed, and zip-bomb. The oversized-event
SDK test now passes.
- **Refactors**
- Hoisted shared test helpers and trimmed comments.
<sup>Written for commit 855cdb052e.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1675?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Session replay batch uploads can now be automatically gzipped (when
supported) to reduce request size; analytics batch uploads use the same
behavior.
* **Bug Fixes**
* Improved handling of gzipped upload bodies, with stricter validation
and safer size/decompression limits to reject invalid payloads and “zip
bomb” scenarios.
* Updated session replay buffering so oversized events are dropped only
when they exceed decompression constraints, while allowing later events
to upload.
* **Tests**
* Added client, server, and end-to-end coverage for gzip/keepalive
behavior and limit enforcement.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
831 lines
30 KiB
TypeScript
831 lines
30 KiB
TypeScript
import { afterEach, describe, expect, it, vi } from "vitest";
|
||
import { KnownErrors } from "../known-errors";
|
||
import { InternalSession, RefreshToken } from "../sessions";
|
||
import { Result } from "../utils/results";
|
||
import { HexclaveClientInterface } from "./client-interface";
|
||
|
||
function createClientInterface(options?: {
|
||
baseUrl?: string,
|
||
apiUrls?: string[],
|
||
probeRate?: number,
|
||
}) {
|
||
const apiUrls = options?.apiUrls ?? [options?.baseUrl ?? "https://api.example.com"];
|
||
return new HexclaveClientInterface({
|
||
clientVersion: "test",
|
||
getBaseUrl: () => apiUrls[0],
|
||
getApiUrls: () => apiUrls,
|
||
probeRate: options?.probeRate,
|
||
extraRequestHeaders: {},
|
||
projectId: "project-id",
|
||
publishableClientKey: "publishable-client-key",
|
||
});
|
||
}
|
||
|
||
function createSession() {
|
||
return new InternalSession({
|
||
refreshAccessTokenCallback: async () => null,
|
||
refreshToken: null,
|
||
accessToken: null,
|
||
});
|
||
}
|
||
|
||
function createJsonResponse(body: unknown): Response {
|
||
return new Response(JSON.stringify(body), {
|
||
status: 200,
|
||
headers: {
|
||
"Content-Type": "application/json",
|
||
},
|
||
});
|
||
}
|
||
|
||
function createKnownErrorResponse(error: InstanceType<typeof KnownErrors[keyof typeof KnownErrors]>): Response {
|
||
return new Response(JSON.stringify({
|
||
code: error.errorCode,
|
||
message: error.message,
|
||
details: error.details,
|
||
}), {
|
||
status: error.statusCode,
|
||
headers: {
|
||
"Content-Type": "application/json",
|
||
"x-stack-known-error": error.errorCode,
|
||
},
|
||
});
|
||
}
|
||
|
||
function createTextResponse(body: string, options: { status: number, headers?: Record<string, string> }): Response {
|
||
return new Response(body, options);
|
||
}
|
||
|
||
function getRequestBody(fetchMock: { mock: { calls: unknown[][] } }): Record<string, unknown> {
|
||
const requestInit = fetchMock.mock.calls[0]?.[1];
|
||
if (requestInit == null || typeof requestInit !== "object" || !("body" in requestInit)) {
|
||
throw new Error("Expected request init to include a body");
|
||
}
|
||
|
||
const requestBody = requestInit.body;
|
||
if (requestBody == null || typeof requestBody !== "string") {
|
||
throw new Error("Expected request body to be a JSON string");
|
||
}
|
||
|
||
const parsed = JSON.parse(requestBody);
|
||
if (parsed == null || typeof parsed !== "object" || Array.isArray(parsed)) {
|
||
throw new Error("Expected parsed request body to be an object");
|
||
}
|
||
|
||
return parsed;
|
||
}
|
||
|
||
afterEach(() => {
|
||
vi.unstubAllGlobals();
|
||
vi.restoreAllMocks();
|
||
});
|
||
|
||
describe("HexclaveClientInterface bot challenge compatibility", () => {
|
||
it("omits bot challenge from magic link requests when no token is provided", async () => {
|
||
const fetchMock = vi.fn(async () => createJsonResponse({ nonce: "nonce" }));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
|
||
const iface = createClientInterface();
|
||
await iface.sendMagicLinkEmail("[email protected]", "https://app.example.com/callback");
|
||
|
||
expect(getRequestBody(fetchMock)).toStrictEqual({
|
||
email: "[email protected]",
|
||
callback_url: "https://app.example.com/callback",
|
||
});
|
||
});
|
||
|
||
it("serializes visible bot challenge retry fields for magic link requests", async () => {
|
||
const fetchMock = vi.fn(async () => createJsonResponse({ nonce: "nonce" }));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
|
||
const iface = createClientInterface();
|
||
await iface.sendMagicLinkEmail("[email protected]", "https://app.example.com/callback", {
|
||
token: " visible-token ",
|
||
phase: "visible",
|
||
});
|
||
|
||
expect(getRequestBody(fetchMock)).toStrictEqual({
|
||
email: "[email protected]",
|
||
callback_url: "https://app.example.com/callback",
|
||
bot_challenge_token: "visible-token",
|
||
bot_challenge_phase: "visible",
|
||
});
|
||
});
|
||
|
||
it("serializes bot challenge unavailability for magic link requests", async () => {
|
||
const fetchMock = vi.fn(async () => createJsonResponse({ nonce: "nonce" }));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
|
||
const iface = createClientInterface();
|
||
await iface.sendMagicLinkEmail("[email protected]", "https://app.example.com/callback", {
|
||
phase: "visible",
|
||
});
|
||
|
||
expect(getRequestBody(fetchMock)).toStrictEqual({
|
||
email: "[email protected]",
|
||
callback_url: "https://app.example.com/callback",
|
||
bot_challenge_unavailable: "true",
|
||
});
|
||
});
|
||
|
||
it("serializes explicit bot challenge unavailability for magic link requests", async () => {
|
||
const fetchMock = vi.fn(async () => createJsonResponse({ nonce: "nonce" }));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
|
||
const iface = createClientInterface();
|
||
await iface.sendMagicLinkEmail("[email protected]", "https://app.example.com/callback", {
|
||
unavailable: true,
|
||
});
|
||
|
||
expect(getRequestBody(fetchMock)).toStrictEqual({
|
||
email: "[email protected]",
|
||
callback_url: "https://app.example.com/callback",
|
||
bot_challenge_unavailable: "true",
|
||
});
|
||
});
|
||
|
||
it("returns BotChallengeFailed as a Result error for magic link requests", async () => {
|
||
const fetchMock = vi.fn(async () => createKnownErrorResponse(
|
||
new KnownErrors.BotChallengeFailed("Visible bot challenge verification failed"),
|
||
));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
|
||
const iface = createClientInterface();
|
||
const result = await iface.sendMagicLinkEmail("[email protected]", "https://app.example.com/callback", {
|
||
phase: "visible",
|
||
});
|
||
|
||
expect(result.status).toBe("error");
|
||
if (result.status !== "error") {
|
||
throw new Error("Expected magic link request to fail with BotChallengeFailed");
|
||
}
|
||
expect(result.error).toBeInstanceOf(KnownErrors.BotChallengeFailed);
|
||
});
|
||
|
||
it("omits bot challenge from credential signup requests when no token is provided", async () => {
|
||
const fetchMock = vi.fn(async () => createJsonResponse({
|
||
access_token: "access-token",
|
||
refresh_token: "refresh-token",
|
||
}));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
|
||
const iface = createClientInterface();
|
||
await iface.signUpWithCredential(
|
||
"[email protected]",
|
||
"password",
|
||
undefined,
|
||
createSession(),
|
||
undefined,
|
||
);
|
||
|
||
expect(getRequestBody(fetchMock)).toStrictEqual({
|
||
email: "[email protected]",
|
||
password: "password",
|
||
});
|
||
});
|
||
|
||
it("returns BotChallengeFailed as a Result error for credential signup requests", async () => {
|
||
const fetchMock = vi.fn(async () => createKnownErrorResponse(
|
||
new KnownErrors.BotChallengeFailed("Visible bot challenge verification failed"),
|
||
));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
|
||
const iface = createClientInterface();
|
||
const result = await iface.signUpWithCredential(
|
||
"[email protected]",
|
||
"password",
|
||
undefined,
|
||
createSession(),
|
||
{
|
||
phase: "visible",
|
||
},
|
||
);
|
||
|
||
expect(result.status).toBe("error");
|
||
if (result.status !== "error") {
|
||
throw new Error("Expected credential signup to fail with BotChallengeFailed");
|
||
}
|
||
expect(result.error).toBeInstanceOf(KnownErrors.BotChallengeFailed);
|
||
});
|
||
|
||
it("omits bot challenge from OAuth URLs when no token is provided", async () => {
|
||
const iface = createClientInterface();
|
||
const oauthUrl = await iface.getOAuthUrl({
|
||
provider: "github",
|
||
redirectUrl: "https://app.example.com/oauth/callback",
|
||
errorRedirectUrl: "https://app.example.com/error",
|
||
codeChallenge: "code-challenge",
|
||
state: "state",
|
||
type: "authenticate",
|
||
session: createSession(),
|
||
});
|
||
|
||
expect(new URL(oauthUrl).searchParams.has("bot_challenge_token")).toBe(false);
|
||
});
|
||
|
||
it("serializes visible bot challenge retry fields in OAuth URLs", async () => {
|
||
const iface = createClientInterface();
|
||
const oauthUrl = await iface.getOAuthUrl({
|
||
provider: "github",
|
||
redirectUrl: "https://app.example.com/oauth/callback",
|
||
errorRedirectUrl: "https://app.example.com/error",
|
||
codeChallenge: "code-challenge",
|
||
state: "state",
|
||
type: "authenticate",
|
||
botChallenge: {
|
||
token: "visible-token",
|
||
phase: "visible",
|
||
},
|
||
session: createSession(),
|
||
});
|
||
|
||
expect(Object.fromEntries(new URL(oauthUrl).searchParams.entries())).toMatchObject({
|
||
bot_challenge_token: "visible-token",
|
||
bot_challenge_phase: "visible",
|
||
});
|
||
});
|
||
|
||
it("serializes bot challenge unavailability in OAuth URLs", async () => {
|
||
const iface = createClientInterface();
|
||
const oauthUrl = await iface.getOAuthUrl({
|
||
provider: "github",
|
||
redirectUrl: "https://app.example.com/oauth/callback",
|
||
errorRedirectUrl: "https://app.example.com/error",
|
||
codeChallenge: "code-challenge",
|
||
state: "state",
|
||
type: "authenticate",
|
||
botChallenge: {
|
||
phase: "visible",
|
||
},
|
||
session: createSession(),
|
||
});
|
||
|
||
expect(Object.fromEntries(new URL(oauthUrl).searchParams.entries())).toMatchObject({
|
||
bot_challenge_unavailable: "true",
|
||
});
|
||
});
|
||
|
||
it("authorizes OAuth via a JSON response instead of relying on manual redirects", async () => {
|
||
const fetchCalls: [input: RequestInfo | URL, init?: RequestInit][] = [];
|
||
const fetchMock = vi.fn(async (input: RequestInfo | URL, init?: RequestInit) => {
|
||
fetchCalls.push([input, init]);
|
||
return createJsonResponse({
|
||
location: "https://accounts.example.com/oauth/authorize",
|
||
});
|
||
});
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
vi.stubGlobal("window", {} as Window & typeof globalThis);
|
||
|
||
const iface = createClientInterface();
|
||
const result = await iface.authorizeOAuth({
|
||
provider: "github",
|
||
redirectUrl: "https://app.example.com/oauth/callback",
|
||
errorRedirectUrl: "https://app.example.com/error",
|
||
codeChallenge: "code-challenge",
|
||
state: "state",
|
||
type: "authenticate",
|
||
session: createSession(),
|
||
});
|
||
|
||
expect(Result.orThrow(result)).toBe("https://accounts.example.com/oauth/authorize");
|
||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||
const [requestUrl, requestInit] = fetchCalls[0] ?? [];
|
||
if (!(typeof requestUrl === "string" || requestUrl instanceof URL)) {
|
||
throw new Error("Expected authorizeOAuth to call fetch with a URL");
|
||
}
|
||
expect(new URL(requestUrl.toString()).searchParams.get("stack_response_mode")).toBe("json");
|
||
expect(requestInit).toMatchObject({
|
||
method: "GET",
|
||
});
|
||
expect(requestInit).not.toHaveProperty("credentials");
|
||
});
|
||
|
||
it("returns BotChallengeFailed as a Result error for OAuth authorization", async () => {
|
||
const fetchMock = vi.fn(async () => createKnownErrorResponse(
|
||
new KnownErrors.BotChallengeFailed("Visible bot challenge verification failed"),
|
||
));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
vi.stubGlobal("window", {} as Window & typeof globalThis);
|
||
|
||
const iface = createClientInterface();
|
||
const result = await iface.authorizeOAuth({
|
||
provider: "github",
|
||
redirectUrl: "https://app.example.com/oauth/callback",
|
||
errorRedirectUrl: "https://app.example.com/error",
|
||
codeChallenge: "code-challenge",
|
||
state: "state",
|
||
type: "authenticate",
|
||
session: createSession(),
|
||
});
|
||
|
||
expect(result.status).toBe("error");
|
||
if (result.status !== "error") {
|
||
throw new Error("Expected OAuth authorization to fail with BotChallengeFailed");
|
||
}
|
||
expect(result.error).toBeInstanceOf(KnownErrors.BotChallengeFailed);
|
||
});
|
||
|
||
it("serializes bot challenge unavailability for credential signup requests", async () => {
|
||
const fetchMock = vi.fn(async () => createJsonResponse({
|
||
access_token: "access-token",
|
||
refresh_token: "refresh-token",
|
||
}));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
|
||
const iface = createClientInterface();
|
||
await iface.signUpWithCredential(
|
||
"[email protected]",
|
||
"password",
|
||
undefined,
|
||
createSession(),
|
||
{
|
||
phase: "visible",
|
||
},
|
||
);
|
||
|
||
expect(getRequestBody(fetchMock)).toStrictEqual({
|
||
email: "[email protected]",
|
||
password: "password",
|
||
bot_challenge_unavailable: "true",
|
||
});
|
||
});
|
||
});
|
||
|
||
describe("_withFallback", () => {
|
||
// ---------------------------------------------------------------------------
|
||
// Helpers — reduce boilerplate across tests
|
||
// ---------------------------------------------------------------------------
|
||
|
||
/** Builds a list of N URL bases: ["https://url-0.test", "https://url-1.test", ...] */
|
||
function urlList(n: number): string[] {
|
||
return Array.from({ length: n }, (_, i) => `https://url-${i}.test`);
|
||
}
|
||
|
||
/** Returns the index of the URL base that `fullUrl` starts with, or -1. */
|
||
function urlIndex(urls: string[], fullUrl: string): number {
|
||
return urls.findIndex(base => fullUrl.startsWith(base));
|
||
}
|
||
|
||
/** Records every fetch URL and calls `handler` to decide the outcome. */
|
||
function mockFetch(handler: (url: string) => "ok" | "fail") {
|
||
const log: string[] = [];
|
||
vi.stubGlobal("fetch", vi.fn(async (input: RequestInfo | URL) => {
|
||
const url = input.toString();
|
||
log.push(url);
|
||
if (handler(url) === "fail") throw new TypeError("Failed to fetch");
|
||
return createJsonResponse({ display_name: "test" });
|
||
}));
|
||
return log;
|
||
}
|
||
|
||
function sendRequest(iface: HexclaveClientInterface) {
|
||
const session = iface.createSession({ refreshToken: null, accessToken: null });
|
||
return iface.sendClientRequest("/users/me", { method: "GET" }, session);
|
||
}
|
||
|
||
// ---------------------------------------------------------------------------
|
||
// Single URL — no fallback
|
||
// ---------------------------------------------------------------------------
|
||
|
||
it("single URL uses standard 5-retry behavior", async () => {
|
||
let attempts = 0;
|
||
vi.stubGlobal("fetch", vi.fn(async () => {
|
||
attempts++;
|
||
if (attempts < 3) throw new TypeError("Failed to fetch");
|
||
return createJsonResponse({ display_name: "test" });
|
||
}));
|
||
|
||
const iface = createClientInterface({ apiUrls: urlList(1) });
|
||
await sendRequest(iface);
|
||
expect(attempts).toBe(3);
|
||
});
|
||
|
||
// ---------------------------------------------------------------------------
|
||
// Normal mode — iterating through URLs in order
|
||
// ---------------------------------------------------------------------------
|
||
|
||
it("uses primary when it is healthy", async () => {
|
||
const urls = urlList(3);
|
||
const log = mockFetch(() => "ok");
|
||
|
||
const iface = createClientInterface({ apiUrls: urls });
|
||
await sendRequest(iface);
|
||
|
||
expect(log.every(u => urlIndex(urls, u) === 0)).toBe(true);
|
||
});
|
||
|
||
it("tries URLs in order and succeeds on first working one", async () => {
|
||
const urls = urlList(4);
|
||
// url-0 and url-1 are down, url-2 is up
|
||
const log = mockFetch((u) => urlIndex(urls, u) < 2 ? "fail" : "ok");
|
||
|
||
const iface = createClientInterface({ apiUrls: urls });
|
||
await sendRequest(iface);
|
||
|
||
expect(urlIndex(urls, log[0])).toBe(0);
|
||
expect(urlIndex(urls, log[1])).toBe(1);
|
||
expect(urlIndex(urls, log[2])).toBe(2);
|
||
expect(log.length).toBe(3);
|
||
});
|
||
|
||
it("does not fall back on KnownError", async () => {
|
||
const urls = urlList(3);
|
||
const log: string[] = [];
|
||
vi.stubGlobal("fetch", vi.fn(async (input: RequestInfo | URL) => {
|
||
log.push(input.toString());
|
||
return createKnownErrorResponse(new KnownErrors.UserNotFound());
|
||
}));
|
||
|
||
const iface = createClientInterface({ apiUrls: urls });
|
||
await expect(sendRequest(iface)).rejects.toThrow();
|
||
expect(log.every(u => urlIndex(urls, u) === 0)).toBe(true);
|
||
});
|
||
|
||
it("does not retry or fall back on non-KnownError 4xx responses", async () => {
|
||
const urls = urlList(3);
|
||
const log: string[] = [];
|
||
vi.stubGlobal("fetch", vi.fn(async (input: RequestInfo | URL) => {
|
||
log.push(input.toString());
|
||
return createTextResponse("Payments are not set up", { status: 402 });
|
||
}));
|
||
|
||
const iface = createClientInterface({ apiUrls: urls });
|
||
await expect(sendRequest(iface)).rejects.toMatchObject({ name: "Error" });
|
||
expect(log.length).toBe(1);
|
||
expect(urlIndex(urls, log[0])).toBe(0);
|
||
});
|
||
|
||
it("wraps non-KnownError 4xx responses as normal errors", async () => {
|
||
const response = createTextResponse("Payments are not set up", { status: 402 });
|
||
vi.stubGlobal("fetch", vi.fn(async () => response));
|
||
|
||
const iface = createClientInterface({ apiUrls: urlList(1) });
|
||
await expect(sendRequest(iface)).rejects.toMatchObject({
|
||
name: "Error",
|
||
message: expect.stringContaining("402 Payments are not set up"),
|
||
cause: response,
|
||
});
|
||
});
|
||
|
||
it("does not retry non-KnownError 5xx responses on a single URL", async () => {
|
||
let attempts = 0;
|
||
vi.stubGlobal("fetch", vi.fn(async () => {
|
||
attempts++;
|
||
return createTextResponse("Server unavailable", { status: 503 });
|
||
}));
|
||
|
||
const iface = createClientInterface({ apiUrls: urlList(1) });
|
||
await expect(sendRequest(iface)).rejects.toThrow("503 Server unavailable");
|
||
expect(attempts).toBe(1);
|
||
});
|
||
|
||
it("falls back on non-KnownError 5xx responses", async () => {
|
||
const urls = urlList(3);
|
||
const log: string[] = [];
|
||
vi.stubGlobal("fetch", vi.fn(async (input: RequestInfo | URL) => {
|
||
const url = input.toString();
|
||
log.push(url);
|
||
if (urlIndex(urls, url) === 0) {
|
||
return createTextResponse("Server unavailable", { status: 503 });
|
||
}
|
||
return createJsonResponse({ display_name: "test" });
|
||
}));
|
||
|
||
const iface = createClientInterface({ apiUrls: urls });
|
||
await sendRequest(iface);
|
||
expect(log.length).toBe(2);
|
||
expect(urlIndex(urls, log[0])).toBe(0);
|
||
expect(urlIndex(urls, log[1])).toBe(1);
|
||
});
|
||
|
||
it("does not fall back on wrapped non-KnownError 4xx refresh token responses", async () => {
|
||
const urls = urlList(3);
|
||
const log: string[] = [];
|
||
vi.stubGlobal("fetch", vi.fn(async (input: RequestInfo | URL) => {
|
||
const url = input instanceof Request ? input.url : input.toString();
|
||
log.push(url);
|
||
return createTextResponse("Payments are not set up", { status: 402 });
|
||
}));
|
||
|
||
const iface = createClientInterface({ apiUrls: urls });
|
||
await expect(iface.fetchNewAccessToken(new RefreshToken("refresh-token"))).rejects.toThrow("Payments are not set up");
|
||
expect(log.length).toBe(1);
|
||
expect(urlIndex(urls, log[0])).toBe(0);
|
||
});
|
||
|
||
it("makes 2 passes × N URLs attempts before throwing", async () => {
|
||
for (const n of [2, 3, 5]) {
|
||
const urls = urlList(n);
|
||
const log = mockFetch(() => "fail");
|
||
|
||
const iface = createClientInterface({ apiUrls: urls });
|
||
await expect(sendRequest(iface)).rejects.toThrow();
|
||
|
||
expect(log.length).toBe(2 * n);
|
||
for (let i = 0; i < n; i++) {
|
||
expect(log.filter(u => urlIndex(urls, u) === i).length).toBe(2);
|
||
}
|
||
}
|
||
});
|
||
|
||
it("bypasses fallback when apiUrlOverride is provided", async () => {
|
||
const log: string[] = [];
|
||
vi.stubGlobal("fetch", vi.fn(async (input: RequestInfo | URL) => {
|
||
log.push(input.toString());
|
||
return createJsonResponse({ display_name: "test" });
|
||
}));
|
||
|
||
const iface = createClientInterface({ apiUrls: urlList(3) });
|
||
const session = iface.createSession({ refreshToken: null, accessToken: null });
|
||
await iface.sendClientRequest("/users/me", { method: "GET" }, session, "client", "https://override.test/api/v1");
|
||
|
||
expect(log.every(u => u.startsWith("https://override.test"))).toBe(true);
|
||
});
|
||
|
||
// ---------------------------------------------------------------------------
|
||
// Sticky mode — remembering the working fallback
|
||
// ---------------------------------------------------------------------------
|
||
|
||
it("enters sticky mode: subsequent requests skip straight to the working fallback", async () => {
|
||
const urls = urlList(4);
|
||
const iface = createClientInterface({ apiUrls: urls, probeRate: 0 });
|
||
|
||
// url-0,1,2 down → sticky on url-3
|
||
mockFetch((u) => urlIndex(urls, u) === 3 ? "ok" : "fail");
|
||
await sendRequest(iface);
|
||
|
||
// Next request goes directly to url-3 (probeRate=0 means no probe)
|
||
const log = mockFetch(() => "ok");
|
||
await sendRequest(iface);
|
||
|
||
expect(log.length).toBe(1);
|
||
expect(urlIndex(urls, log[0])).toBe(3);
|
||
});
|
||
|
||
it("probes primary and exits sticky mode when probe succeeds", async () => {
|
||
const urls = urlList(3);
|
||
const iface = createClientInterface({ apiUrls: urls, probeRate: 1 });
|
||
|
||
// url-0 down → sticky on url-1
|
||
mockFetch((u) => urlIndex(urls, u) === 0 ? "fail" : "ok");
|
||
await sendRequest(iface);
|
||
|
||
// url-0 recovers. probeRate=1 → always probes → probe succeeds → exits sticky
|
||
const log = mockFetch(() => "ok");
|
||
await sendRequest(iface);
|
||
expect(urlIndex(urls, log[0])).toBe(0);
|
||
|
||
// Next request should go to url-0 directly (no longer sticky)
|
||
const log2 = mockFetch(() => "ok");
|
||
await sendRequest(iface);
|
||
expect(log2.length).toBe(1);
|
||
expect(urlIndex(urls, log2[0])).toBe(0);
|
||
});
|
||
|
||
it("halves probe rate on each failed probe", async () => {
|
||
const urls = urlList(2);
|
||
const iface = createClientInterface({ apiUrls: urls, probeRate: 1 });
|
||
|
||
// Enter sticky on url-1, url-0 stays permanently down
|
||
mockFetch((u) => urlIndex(urls, u) === 0 ? "fail" : "ok");
|
||
await sendRequest(iface);
|
||
|
||
// probeRate=1 → probes url-0, fails → rate becomes 0.5
|
||
mockFetch((u) => urlIndex(urls, u) === 0 ? "fail" : "ok");
|
||
await sendRequest(iface);
|
||
|
||
// probeRate=0.5 → probes (random < 0.5), fails → rate becomes 0.25
|
||
const randomMock = vi.spyOn(Math, "random").mockReturnValue(0.4);
|
||
mockFetch((u) => urlIndex(urls, u) === 0 ? "fail" : "ok");
|
||
await sendRequest(iface);
|
||
|
||
// rate=0.25, random=0.3 ≥ 0.25 → should NOT probe primary
|
||
randomMock.mockReturnValue(0.3);
|
||
const log = mockFetch(() => "ok");
|
||
await sendRequest(iface);
|
||
|
||
expect(log.length).toBe(1);
|
||
expect(urlIndex(urls, log[0])).toBe(1);
|
||
|
||
randomMock.mockRestore();
|
||
});
|
||
|
||
// ---------------------------------------------------------------------------
|
||
// Sticky URL goes down — falls through to full iteration
|
||
// ---------------------------------------------------------------------------
|
||
|
||
it("falls through to full iteration when sticky URL also goes down", async () => {
|
||
const urls = urlList(3);
|
||
const iface = createClientInterface({ apiUrls: urls, probeRate: 0 });
|
||
|
||
// url-0,1 down → sticky on url-2
|
||
mockFetch((u) => urlIndex(urls, u) === 2 ? "ok" : "fail");
|
||
await sendRequest(iface);
|
||
|
||
// Now url-2 also down, url-1 recovers
|
||
const log = mockFetch((u) => urlIndex(urls, u) === 1 ? "ok" : "fail");
|
||
await sendRequest(iface);
|
||
|
||
// Should have tried sticky url-2 (failed), then iterated 0→1 (found url-1)
|
||
expect(log.some(u => urlIndex(urls, u) === 2)).toBe(true);
|
||
expect(log.some(u => urlIndex(urls, u) === 1)).toBe(true);
|
||
});
|
||
|
||
it("re-enters sticky on the new working URL after fallthrough", async () => {
|
||
const urls = urlList(4);
|
||
const iface = createClientInterface({ apiUrls: urls, probeRate: 0 });
|
||
|
||
// sticky on url-3
|
||
mockFetch((u) => urlIndex(urls, u) === 3 ? "ok" : "fail");
|
||
await sendRequest(iface);
|
||
|
||
// url-3 dies, url-2 recovers → should re-sticky on url-2
|
||
mockFetch((u) => urlIndex(urls, u) === 2 ? "ok" : "fail");
|
||
await sendRequest(iface);
|
||
|
||
// Next request goes directly to url-2
|
||
const log = mockFetch(() => "ok");
|
||
await sendRequest(iface);
|
||
|
||
expect(log.length).toBe(1);
|
||
expect(urlIndex(urls, log[0])).toBe(2);
|
||
});
|
||
|
||
it("throws when sticky URL fails and all URLs fail in iteration", async () => {
|
||
const urls = urlList(3);
|
||
const iface = createClientInterface({ apiUrls: urls, probeRate: 0 });
|
||
|
||
// sticky on url-1
|
||
mockFetch((u) => urlIndex(urls, u) === 1 ? "ok" : "fail");
|
||
await sendRequest(iface);
|
||
|
||
// Everything is now down
|
||
const log = mockFetch(() => "fail");
|
||
await expect(sendRequest(iface)).rejects.toThrow();
|
||
|
||
// sticky attempt (1) + 2 passes × 3 URLs (6) = 7
|
||
expect(log.length).toBe(7);
|
||
});
|
||
|
||
it("does not probe primary when sticky URL fails (probe only before sticky attempt)", async () => {
|
||
const urls = urlList(3);
|
||
const iface = createClientInterface({ apiUrls: urls, probeRate: 1 });
|
||
|
||
// sticky on url-2, url-0 stays down
|
||
mockFetch((u) => urlIndex(urls, u) === 2 ? "ok" : "fail");
|
||
await sendRequest(iface);
|
||
|
||
// url-0 still down, url-2 also dies, url-1 is the only one up
|
||
// probeRate=1 → probes url-0 first (fails), then tries sticky url-2 (fails),
|
||
// then full iteration finds url-1
|
||
const log = mockFetch((u) => urlIndex(urls, u) === 1 ? "ok" : "fail");
|
||
await sendRequest(iface);
|
||
|
||
const hitOrder = log.map(u => urlIndex(urls, u));
|
||
// probe url-0, sticky url-2, then iteration: 0, 1 (succeeds)
|
||
expect(hitOrder[0]).toBe(0); // probe
|
||
expect(hitOrder[1]).toBe(2); // sticky attempt
|
||
expect(hitOrder).toContain(1); // found during iteration
|
||
});
|
||
});
|
||
|
||
function getRequestInit(fetchMock: { mock: { calls: unknown[][] } }): RequestInit {
|
||
const init = fetchMock.mock.calls[0]?.[1];
|
||
if (init == null || typeof init !== "object") throw new Error("expected RequestInit");
|
||
return init as RequestInit;
|
||
}
|
||
|
||
async function gunzipToText(body: unknown): Promise<string> {
|
||
if (!(body instanceof Uint8Array)) throw new Error("expected Uint8Array body");
|
||
const stream = new Blob([body as BlobPart]).stream().pipeThrough(new DecompressionStream("gzip"));
|
||
return await new Response(stream).text();
|
||
}
|
||
|
||
describe("sendAnalyticsEventBatch encoding", () => {
|
||
function captureFetch() {
|
||
const fetchMock = vi.fn(async () => createJsonResponse({ inserted: 0 }));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
return fetchMock;
|
||
}
|
||
|
||
it("gzips body and sends application/octet-stream when keepalive is false", async () => {
|
||
const fetchMock = captureFetch();
|
||
const iface = createClientInterface();
|
||
const payload = JSON.stringify({ batch_id: "abc", events: [{ event_type: "$click" }] });
|
||
|
||
await iface.sendAnalyticsEventBatch(payload, null, { keepalive: false });
|
||
|
||
const init = getRequestInit(fetchMock);
|
||
const contentType = new Headers(init.headers).get("Content-Type");
|
||
expect(contentType).toBe("application/octet-stream");
|
||
expect(init.body).toBeInstanceOf(Uint8Array);
|
||
await expect(gunzipToText(init.body)).resolves.toBe(payload);
|
||
});
|
||
|
||
it("falls back to plain JSON when keepalive is true (avoids racing pagehide tear-down)", async () => {
|
||
const fetchMock = captureFetch();
|
||
const iface = createClientInterface();
|
||
const payload = JSON.stringify({ batch_id: "abc", events: [] });
|
||
|
||
await iface.sendAnalyticsEventBatch(payload, null, { keepalive: true });
|
||
|
||
const init = getRequestInit(fetchMock);
|
||
expect(new Headers(init.headers).get("Content-Type")).toBe("application/json");
|
||
expect(init.body).toBe(payload);
|
||
});
|
||
|
||
it("falls back to plain JSON when CompressionStream is unavailable", async () => {
|
||
vi.stubGlobal("CompressionStream", undefined);
|
||
const fetchMock = captureFetch();
|
||
const iface = createClientInterface();
|
||
const payload = JSON.stringify({ batch_id: "abc", events: [] });
|
||
|
||
await iface.sendAnalyticsEventBatch(payload, null, { keepalive: false });
|
||
|
||
const init = getRequestInit(fetchMock);
|
||
expect(new Headers(init.headers).get("Content-Type")).toBe("application/json");
|
||
expect(init.body).toBe(payload);
|
||
});
|
||
|
||
it("falls back to plain JSON when CompressionStream throws at runtime", async () => {
|
||
class ThrowingCompressionStream {
|
||
constructor() { throw new Error("compression unsupported"); }
|
||
}
|
||
vi.stubGlobal("CompressionStream", ThrowingCompressionStream);
|
||
const fetchMock = captureFetch();
|
||
const iface = createClientInterface();
|
||
const payload = JSON.stringify({ batch_id: "abc", events: [] });
|
||
|
||
await iface.sendAnalyticsEventBatch(payload, null, { keepalive: false });
|
||
|
||
const init = getRequestInit(fetchMock);
|
||
expect(new Headers(init.headers).get("Content-Type")).toBe("application/json");
|
||
expect(init.body).toBe(payload);
|
||
});
|
||
});
|
||
|
||
describe("sendSessionReplayBatch encoding", () => {
|
||
function captureFetch() {
|
||
const fetchMock = vi.fn(async () => createJsonResponse({ session_replay_id: "r", batch_id: "b", s3_key: "k", deduped: false }));
|
||
vi.stubGlobal("fetch", fetchMock);
|
||
return fetchMock;
|
||
}
|
||
|
||
it("gzips body and sends application/octet-stream when keepalive is false", async () => {
|
||
const fetchMock = captureFetch();
|
||
const iface = createClientInterface();
|
||
// A large, highly-compressible payload — the case the wire-limit drop used to lose.
|
||
const payload = JSON.stringify({ batch_id: "abc", events: [{ type: 2, data: "x".repeat(1_000_000) }] });
|
||
|
||
await iface.sendSessionReplayBatch(payload, null, { keepalive: false });
|
||
|
||
const init = getRequestInit(fetchMock);
|
||
expect(new Headers(init.headers).get("Content-Type")).toBe("application/octet-stream");
|
||
expect(init.body).toBeInstanceOf(Uint8Array);
|
||
// The compressed body is far smaller than the 1MB+ raw payload.
|
||
expect((init.body as Uint8Array).byteLength).toBeLessThan(100_000);
|
||
await expect(gunzipToText(init.body)).resolves.toBe(payload);
|
||
});
|
||
|
||
it("falls back to plain JSON when keepalive is true (avoids racing pagehide tear-down)", async () => {
|
||
const fetchMock = captureFetch();
|
||
const iface = createClientInterface();
|
||
const payload = JSON.stringify({ batch_id: "abc", events: [] });
|
||
|
||
await iface.sendSessionReplayBatch(payload, null, { keepalive: true });
|
||
|
||
const init = getRequestInit(fetchMock);
|
||
expect(new Headers(init.headers).get("Content-Type")).toBe("application/json");
|
||
expect(init.body).toBe(payload);
|
||
});
|
||
|
||
it("falls back to plain JSON when CompressionStream is unavailable", async () => {
|
||
vi.stubGlobal("CompressionStream", undefined);
|
||
const fetchMock = captureFetch();
|
||
const iface = createClientInterface();
|
||
const payload = JSON.stringify({ batch_id: "abc", events: [] });
|
||
|
||
await iface.sendSessionReplayBatch(payload, null, { keepalive: false });
|
||
|
||
const init = getRequestInit(fetchMock);
|
||
expect(new Headers(init.headers).get("Content-Type")).toBe("application/json");
|
||
expect(init.body).toBe(payload);
|
||
});
|
||
|
||
it("falls back to plain JSON when CompressionStream throws at runtime", async () => {
|
||
class ThrowingCompressionStream {
|
||
constructor() { throw new Error("compression unsupported"); }
|
||
}
|
||
vi.stubGlobal("CompressionStream", ThrowingCompressionStream);
|
||
const fetchMock = captureFetch();
|
||
const iface = createClientInterface();
|
||
const payload = JSON.stringify({ batch_id: "abc", events: [] });
|
||
|
||
await iface.sendSessionReplayBatch(payload, null, { keepalive: false });
|
||
|
||
const init = getRequestInit(fetchMock);
|
||
expect(new Headers(init.headers).get("Content-Type")).toBe("application/json");
|
||
expect(init.body).toBe(payload);
|
||
});
|
||
});
|