From 2f0d9bc6cb2190ef597bc1edf41334556eef1882 Mon Sep 17 00:00:00 2001 From: BilalG1 Date: Tue, 30 Jun 2026 13:13:11 -0700 Subject: [PATCH] fix(session-replay): gzip replay batches so large rrweb snapshots aren't dropped (#1675) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Problem Session replay batches are uploaded as **raw, uncompressed JSON**. A single large rrweb full snapshot (>900 KB) is dropped client-side in `SessionRecorder._flush`, and any batch near the server's 1 MB body limit is rejected. Meanwhile the **analytics events** path right next door already gzips on the wire (`encodeAnalyticsBody` + a server-side gunzip with zip-bomb guards) — replays just never got the same treatment. Notably, `dev` was already **red** on the SDK test `sends a single oversized event alone without dropping it` (the code dropped at 900 KB while the test expected the event to be sent). This change makes that test pass. ## What this does Ports the proven analytics compression pattern to session replays: - **`packages/shared` (client transport):** generalize `encodeAnalyticsBody` → `encodeGzipJsonBody` and wire it into `sendSessionReplayBatch`. Replays now gzip via native `CompressionStream` (`application/octet-stream`), with the same fallbacks: keepalive flushes send plain JSON (so the request survives page teardown), and a missing/throwing `CompressionStream` (Safari < 16.4) degrades to plain JSON. - **`packages/template` (SDK recorder):** replace the 900 KB hard-drop with `MAX_SINGLE_EVENT_BYTES = 8 MB`, kept in sync with the server's decompressed cap. The wire is gzipped downstream, so an oversized single event is now sent compressed instead of discarded; only an event that could never fit *decompressed* is dropped. - **`apps/backend` (route):** add `maybeDecodeBinaryBody` (gunzip with a 1 MB compressed cap + 8 MB `maxOutputLength` zip-bomb guard) via a `.transform` on the batch route body schema. Plain-JSON bodies pass through untouched. ## Why gzip / CompressionStream Benchmarked on a real rrweb capture (Puppeteer, rrweb 1.1.3): a **1.09 MiB full snapshot → 134 KiB gzip (~8×) in ~6.6 ms**. Native `CompressionStream` adds **0 bundle bytes** and was faster than fflate. This mirrors what PostHog (gzip, native `CompressionStream` + fflate fallback) and Sentry (fflate deflate in a worker) both ship — both rely on compression, not splitting, as the primary mechanism for large events. ## Tests - **Client SDK** (`client-interface.test.ts`): gzip+octet-stream on non-keepalive (round-trip + size assert), plain JSON on keepalive, and both `CompressionStream`-unavailable fallbacks. - **SDK recorder** (`session-replay.test.ts`): existing oversized-event test now passes; new test that a >8 MB event is dropped while a following normal event still sends. - **Backend e2e** (`session-replays.test.ts`): gzipped happy path (`compressed < 1MB < raw`), invalid gzip → 400, oversized compressed → 413, zip-bomb → 400. Verified passing against a live local backend. ## Verification notes - `@hexclave/shared` + `@hexclave/template` typecheck pass; touched files lint clean; SDK unit tests pass. - Two **pre-existing, unrelated** local failures (each confirmed by stashing this change and re-running on clean `dev`): a backend typecheck error in `oauth/ssrf-protection.test.ts` (`NODE_ENV` read-only), and 3 session-replay **quota** e2e tests failing in the payments `setSessionReplayItemQuantity` setup helper. ## Follow-up (out of scope) The keepalive/unload path still sends uncompressed (64 KB browser keepalive cap), so a very large final batch on page exit can still be lost — not addressed here. --- ## Summary by cubic Gzip session replay batches so large rrweb snapshots are sent instead of dropped. Keepalive flushes cap single-event size to avoid 413s on page unload, and limits now include a 1 KiB envelope margin to match server caps. - **Bug Fixes** - Client (`@hexclave/shared`): generalized `encodeAnalyticsBody` → `encodeGzipJsonBody`; gzip via native `CompressionStream` for `/session-replays/batch`; keepalive and missing/throwing `CompressionStream` fall back to JSON. - SDK recorder (`@hexclave/template`): normal flushes allow a single event up to `MAX_SINGLE_EVENT_BYTES = 8 MiB - 1 KiB`; keepalive flushes cap single-event size at `MAX_BATCH_UNCOMPRESSED_BYTES = 900 KB`; send oversized events alone and rely on gzip; drop only above those thresholds; log a distinct 413 warning with the count of buffered events dropped and stop the loop. - Backend: gunzip `application/octet-stream` bodies with caps (`1 MB` compressed, `8 MiB` decompressed); invalid gzip → 400; oversized compressed → 413; plain JSON passes through. - Tests: added client encoding tests for session replays; SDK tests for >8 MiB drop, keepalive drop, and 413 warning; backend e2e for gzip, invalid gzip, oversized compressed, and zip-bomb. The oversized-event SDK test now passes. - **Refactors** - Hoisted shared test helpers and trimmed comments. Written for commit 855cdb052eac5c0abd4166d7157273ee53136009. Summary will update on new commits. Review in cubic ## Summary by CodeRabbit * **New Features** * Session replay batch uploads can now be automatically gzipped (when supported) to reduce request size; analytics batch uploads use the same behavior. * **Bug Fixes** * Improved handling of gzipped upload bodies, with stricter validation and safer size/decompression limits to reject invalid payloads and “zip bomb” scenarios. * Updated session replay buffering so oversized events are dropped only when they exceed decompression constraints, while allowing later events to upload. * **Tests** * Added client, server, and end-to-end coverage for gzip/keepalive behavior and limit enforcement. --- .../latest/session-replays/batch/route.tsx | 33 +++- .../endpoints/api/v1/session-replays.test.ts | 109 +++++++++++- .../src/interface/client-interface.test.ts | 89 ++++++++-- .../shared/src/interface/client-interface.ts | 18 +- .../implementations/session-replay.test.ts | 161 +++++++++++++++++- .../apps/implementations/session-replay.ts | 50 ++++-- 6 files changed, 423 insertions(+), 37 deletions(-) diff --git a/apps/backend/src/app/api/latest/session-replays/batch/route.tsx b/apps/backend/src/app/api/latest/session-replays/batch/route.tsx index 02e3798fd..2c5955384 100644 --- a/apps/backend/src/app/api/latest/session-replays/batch/route.tsx +++ b/apps/backend/src/app/api/latest/session-replays/batch/route.tsx @@ -11,7 +11,7 @@ import { adaptSchema, clientOrHigherAuthTypeSchema, yupArray, yupMixed, yupNumbe import { StatusError } from "@hexclave/shared/dist/utils/errors"; import { randomUUID } from "node:crypto"; import { promisify } from "node:util"; -import { gzip as gzipCb } from "node:zlib"; +import { gzip as gzipCb, gunzipSync } from "node:zlib"; const gzip = promisify(gzipCb); @@ -19,6 +19,35 @@ const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[089ab][0-9a-f]{3}-[0 const MAX_BODY_BYTES = 1_000_000; const MAX_EVENTS = 5_000; +// Zip-bomb guard; keep in sync with the client's MAX_SINGLE_EVENT_BYTES. +const MAX_DECOMPRESSED_BYTES = 8 * 1024 * 1024; + +// Gunzips application/octet-stream bodies (the client gzips large batches); +// plain application/json bodies pass through untouched. +function maybeDecodeBinaryBody(value: unknown): unknown { + let bytes: Uint8Array | undefined; + if (value instanceof ArrayBuffer) { + bytes = new Uint8Array(value); + } else if (value instanceof Uint8Array) { + bytes = value; + } + if (!bytes) return value; + + if (bytes.byteLength > MAX_BODY_BYTES) { + throw new StatusError(StatusError.PayloadTooLarge, `Encoded session replay body too large (max ${MAX_BODY_BYTES} bytes)`); + } + let decompressed: Buffer; + try { + decompressed = gunzipSync(bytes, { maxOutputLength: MAX_DECOMPRESSED_BYTES }); + } catch { + throw new StatusError(StatusError.BadRequest, "Invalid encoded session replay body"); + } + try { + return JSON.parse(decompressed.toString("utf-8")); + } catch { + throw new StatusError(StatusError.BadRequest, "Invalid encoded session replay body"); + } +} function extractEventTimesMs(events: unknown[], fallbackMs: number) { let minTs = Infinity; @@ -60,7 +89,7 @@ export const POST = createSmartRouteHandler({ started_at_ms: yupNumber().defined().integer().min(0), sent_at_ms: yupNumber().defined().integer().min(0), events: yupArray(yupMixed().defined()).defined(), - }).defined(), + }).defined().transform((_value, originalValue) => maybeDecodeBinaryBody(originalValue)), }), response: yupObject({ statusCode: yupNumber().oneOf([200]).defined(), diff --git a/apps/e2e/tests/backend/endpoints/api/v1/session-replays.test.ts b/apps/e2e/tests/backend/endpoints/api/v1/session-replays.test.ts index 351d2f6b4..abaccc8ad 100644 --- a/apps/e2e/tests/backend/endpoints/api/v1/session-replays.test.ts +++ b/apps/e2e/tests/backend/endpoints/api/v1/session-replays.test.ts @@ -1,4 +1,5 @@ -import { randomUUID } from "node:crypto"; +import { randomBytes, randomUUID } from "node:crypto"; +import { gzipSync } from "node:zlib"; import { PLAN_LIMITS } from "@hexclave/shared/dist/plans"; import { wait } from "@hexclave/shared/dist/utils/promises"; import { it } from "../../../../helpers"; @@ -162,6 +163,112 @@ it("stores session replay batch metadata and dedupes by (session_replay_id, batc expect(second.body?.session_replay_id).toBe(recordingId); }); +it("accepts a gzipped binary body (compressed large-payload encoding)", async ({ expect }) => { + await Project.createAndSwitch({ config: { magic_link_enabled: true } }); + await Project.updateConfig({ apps: { installed: { analytics: { enabled: true } } } }); + await Auth.Otp.signIn(); + + const now = Date.now(); + const payload = { + browser_session_id: randomUUID(), + session_replay_segment_id: randomUUID(), + batch_id: randomUUID(), + started_at_ms: now, + sent_at_ms: now + 500, + // Large full snapshot: exceeds the 1MB raw wire limit but gzips under it. + events: [{ timestamp: now + 100, type: 2, data: { html: "x".repeat(2_000_000) } }], + }; + const compressed = gzipSync(Buffer.from(JSON.stringify(payload), "utf-8")); + // Sanity: the raw payload exceeds the wire limit, the compressed one doesn't. + expect(compressed.byteLength).toBeLessThan(1_000_000); + + const res = await niceBackendFetch("/api/v1/session-replays/batch", { + method: "POST", + accessType: "client", + rawBody: compressed, + }); + + expect(res).toMatchInlineSnapshot(` + NiceResponse { + "status": 200, + "body": { + "batch_id": "", + "deduped": false, + "s3_key": "session-replays//main//.json.gz", + "session_replay_id": "", + }, + "headers": Headers {