2f0d9bc6cb
## Problem
Session replay batches are uploaded as **raw, uncompressed JSON**. A
single large rrweb full snapshot (>900 KB) is dropped client-side in
`SessionRecorder._flush`, and any batch near the server's 1 MB body
limit is rejected. Meanwhile the **analytics events** path right next
door already gzips on the wire (`encodeAnalyticsBody` + a server-side
gunzip with zip-bomb guards) — replays just never got the same
treatment.
Notably, `dev` was already **red** on the SDK test `sends a single
oversized event alone without dropping it` (the code dropped at 900 KB
while the test expected the event to be sent). This change makes that
test pass.
## What this does
Ports the proven analytics compression pattern to session replays:
- **`packages/shared` (client transport):** generalize
`encodeAnalyticsBody` → `encodeGzipJsonBody` and wire it into
`sendSessionReplayBatch`. Replays now gzip via native
`CompressionStream` (`application/octet-stream`), with the same
fallbacks: keepalive flushes send plain JSON (so the request survives
page teardown), and a missing/throwing `CompressionStream` (Safari <
16.4) degrades to plain JSON.
- **`packages/template` (SDK recorder):** replace the 900 KB hard-drop
with `MAX_SINGLE_EVENT_BYTES = 8 MB`, kept in sync with the server's
decompressed cap. The wire is gzipped downstream, so an oversized single
event is now sent compressed instead of discarded; only an event that
could never fit *decompressed* is dropped.
- **`apps/backend` (route):** add `maybeDecodeBinaryBody` (gunzip with a
1 MB compressed cap + 8 MB `maxOutputLength` zip-bomb guard) via a
`.transform` on the batch route body schema. Plain-JSON bodies pass
through untouched.
## Why gzip / CompressionStream
Benchmarked on a real rrweb capture (Puppeteer, rrweb 1.1.3): a **1.09
MiB full snapshot → 134 KiB gzip (~8×) in ~6.6 ms**. Native
`CompressionStream` adds **0 bundle bytes** and was faster than fflate.
This mirrors what PostHog (gzip, native `CompressionStream` + fflate
fallback) and Sentry (fflate deflate in a worker) both ship — both rely
on compression, not splitting, as the primary mechanism for large
events.
## Tests
- **Client SDK** (`client-interface.test.ts`): gzip+octet-stream on
non-keepalive (round-trip + size assert), plain JSON on keepalive, and
both `CompressionStream`-unavailable fallbacks.
- **SDK recorder** (`session-replay.test.ts`): existing oversized-event
test now passes; new test that a >8 MB event is dropped while a
following normal event still sends.
- **Backend e2e** (`session-replays.test.ts`): gzipped happy path
(`compressed < 1MB < raw`), invalid gzip → 400, oversized compressed →
413, zip-bomb → 400. Verified passing against a live local backend.
## Verification notes
- `@hexclave/shared` + `@hexclave/template` typecheck pass; touched
files lint clean; SDK unit tests pass.
- Two **pre-existing, unrelated** local failures (each confirmed by
stashing this change and re-running on clean `dev`): a backend typecheck
error in `oauth/ssrf-protection.test.ts` (`NODE_ENV` read-only), and 3
session-replay **quota** e2e tests failing in the payments
`setSessionReplayItemQuantity` setup helper.
## Follow-up (out of scope)
The keepalive/unload path still sends uncompressed (64 KB browser
keepalive cap), so a very large final batch on page exit can still be
lost — not addressed here.
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Gzip session replay batches so large rrweb snapshots are sent instead of
dropped. Keepalive flushes cap single-event size to avoid 413s on page
unload, and limits now include a 1 KiB envelope margin to match server
caps.
- **Bug Fixes**
- Client (`@hexclave/shared`): generalized `encodeAnalyticsBody` →
`encodeGzipJsonBody`; gzip via native `CompressionStream` for
`/session-replays/batch`; keepalive and missing/throwing
`CompressionStream` fall back to JSON.
- SDK recorder (`@hexclave/template`): normal flushes allow a single
event up to `MAX_SINGLE_EVENT_BYTES = 8 MiB - 1 KiB`; keepalive flushes
cap single-event size at `MAX_BATCH_UNCOMPRESSED_BYTES = 900 KB`; send
oversized events alone and rely on gzip; drop only above those
thresholds; log a distinct 413 warning with the count of buffered events
dropped and stop the loop.
- Backend: gunzip `application/octet-stream` bodies with caps (`1 MB`
compressed, `8 MiB` decompressed); invalid gzip → 400; oversized
compressed → 413; plain JSON passes through.
- Tests: added client encoding tests for session replays; SDK tests for
>8 MiB drop, keepalive drop, and 413 warning; backend e2e for gzip,
invalid gzip, oversized compressed, and zip-bomb. The oversized-event
SDK test now passes.
- **Refactors**
- Hoisted shared test helpers and trimmed comments.
<sup>Written for commit
|
||
|---|---|---|
| .. | ||
| src | ||
| .eslintrc.cjs | ||
| LICENSE | ||
| package.json | ||
| README.md | ||
| tsconfig.json | ||
| tsdown.config.ts | ||
| vitest.config.ts | ||
Hexclave
The user infrastructure platform.
Hexclave handles everything around your users: authentication, teams, payments, emails, analytics, and much more. Start in minutes on the hosted cloud. Your data is always yours to export and self-host.
Website · Docs · Dashboard · Discord
Get started
Setting up Hexclave is one prompt. Paste this into your coding agent of choice:
Read skill.hexclave.com and help me setup hexclave in this project
For AI agents
If you're an AI coding agent, use your web fetch tool or curl the skill documentation for up-to-date integration instructions:
curl -sSL "https://skill.hexclave.com?question=<your-question>&context=<your-context>"
What's included
Hexclave ships as a catalog of apps you switch on as your product needs them. Each one is built on the same user model, and new apps land regularly.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Contributing
Hexclave is open source, and contributions are welcome. Read
CONTRIBUTING.md to get started, and say hello in
Discord before picking up anything large.
Found a security issue? Email security@hexclave.com.