stack/packages/shared/src/utils/esbuild.tsx
Mantra 5209ec83ae
Migrate backend transport from Next.js to ElysiaJS (WIP) (#1630)
> **Draft / WIP — do not merge.** Backend Next.js → ElysiaJS transport
migration, in progress.

## Summary

Migrates the backend (`apps/backend`) HTTP transport from **Next.js (App
Router)** to **ElysiaJS** (Node adapter, `@elysiajs/node`), with the
explicit goal of keeping every API route **byte-for-byte backwards
compatible** — same URLs, methods, status codes, headers, and bodies.
Elysia becomes purely the transport layer; the existing
`createSmartRouteHandler` abstraction and all ~221 file-based route
handlers are reused unchanged.

## Approach

- **One wildcard dispatcher** reuses the existing `smart-router`
matchers + generated `routes.json`/`api-versions.json` rather than
re-registering 221 routes on Elysia's router.
- **`proxy.tsx` (Next 16 middleware) ported** to an Elysia request
pipeline: CORS, dev rate-limit, `x-hexclave-*`→`x-stack-*` header
aliasing, OPTIONS preflight, and the **API version rewrite** (`/api/v1`,
`/api/v2betaN` → `/api/latest` / `/api/migrations/*`).
- **Two-URL dispatch**: the version-rewritten path locates the handler;
the original client URL is preserved on `req.url`/`nextUrl`
(load-bearing for OIDC/neon routes that assert `/api/v1/...`).
- **`next/*` compatibility shims** (`next/headers`, `next/navigation`,
`next/server`) under `src/lib/next-compat/`, wired via aliases in
tsconfig / vitest / tsdown so route files stay untouched.
- **Observability** moved off `@sentry/nextjs` → `@sentry/node` + manual
OpenTelemetry NodeSDK preload (`src/instrument.ts`); explicit
`/monitoring` Sentry tunnel.
- **Build/deploy**: `next build/start` → tsdown bundle
(`tsdown.config.ts`) for container; Vercel default-export entry;
Dockerfile CMD → `dist/server.mjs`; `next.config.mjs` removed (security
headers etc. reimplemented).

## Status — WIP

**Green so far:** backend `typecheck`, backend `lint`, tsdown bundle,
`GET /health`, `GET /api/v1`, e2e `migration-tests` (13/13),
`analytics-query` (69/69), unit suite (~1006 tests).

**Remaining before ready for review:**
- [ ] Full backend e2e snapshot suite green (snapshots = byte-for-byte
oracle; never `-u`'d to mask drift)
- [ ] `config.test.ts` (`custom_oidc`) snapshot root-cause
- [ ] M4: source-map upload + remove remaining `@sentry/nextjs` browser
leftovers
- [ ] M5: Docker + Vercel artifact verification
- [ ] M6: confirm React-UI → slim-handler cleanup

## Notes
- Backwards-compat gate: the existing e2e snapshot files (recorded
against the Next.js backend) must pass unchanged.
- Built primarily by Codex (`gpt-5.5`); branch intentionally separate
from `dev`.

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Migrates the backend transport from Next.js to ElysiaJS with
byte‑for‑byte compatible APIs, bundled with `tsdown` for Node, Vercel,
and Docker. Reverts out‑of‑scope payments/metrics/custom OIDC changes to
keep this PR focused.

- **Refactors**
- Transport: Next.js → ElysiaJS (`elysia`, `@elysiajs/node`) via
wildcard dispatcher + generated route registry; preserve original URL;
decode params; 400 on malformed params.
- Next shims: `next/headers`, `next/navigation`, `next/server` with
request context and cookie serialization; don’t percent‑decode;
`cookies().delete()` clears pending Set‑Cookie; add `fetch` `next`
options typing.
- Observability: move to `@sentry/node`/`@sentry/browser`; preload
`@opentelemetry/sdk-node`; guard duplicate OTel registration; add
`/monitoring` tunnel; sanitize Sentry release names; replace Next.js
instrumentation with `src/instrument.ts`.
- Build/deploy: bundle `dist/server.mjs` and `dist/vercel.mjs` with
`tsdown`; Vercel function at `api/index.ts` re‑exports the handler
(`runtime: nodejs`, `maxDuration: 60`, `framework: null` rewrite);
Docker runs the Node bundle.
- Runtime: reimplement security headers; dev rate limiter uses high‑res
timers; graceful SIGTERM.
- Env/dev: expand nested env refs; load `.env.development` in dev;
TS/Vitest alias `next/*` to shims.
- Deps/tooling: re‑add `@sinclair/typebox`; regenerate `pnpm-lock.yaml`
with pnpm 11.5.0 and pin `exact-mirror@1.1.1`.

- **Bug Fixes**
  - Dispatcher: catch `NextNotFoundError` and return 404.
- Emails: deterministic localhost/loopback SMTP fallback (prefer IPv6)
and consistent HTML for snapshot parity.
- E2E parity: restore email‑conflict error; poll all‑users outbox before
asserting; extend refund/transactions timeouts; fix team invitation
revoke setup.
- Build/Docker: prevent `dist` wipes by setting `clean:false` in
db‑migrations `tsdown` config; reorder COPY so `dist/server.mjs` is
present; backend entrypoint uses `dist/server.mjs`.
- CI/tests: Vitest `minWorkers: 1`; fallback e2e starts the Elysia
bundle via `pnpm run start` with `PORT`.
- Shared: normalize `esbuild-wasm` default export under Node to avoid
runtime mismatches.

<sup>Written for commit d3c9b0ff22.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/hexclave/hexclave/pull/1630?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **Bug Fixes**
* Improved email delivery by adding localhost/loopback-aware SMTP retry
behavior and more consistent delivered email HTML.
  * Enhanced error reporting during external database synchronization.
* Refined OAuth provider type validation to better handle missing or
custom provider configurations.

* **Improvements**
* Updated backend runtime and routing/request handling for more
consistent behavior, including updated Vercel/Docker startup.
* Improved payment “dual write” reliability by scheduling Bulldozer
projection updates with per-tenant ordering.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mantra <mantra@stack-auth.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bilal Godil <bg2002@gmail.com>
2026-07-09 21:39:28 -07:00

251 lines
9.9 KiB
TypeScript
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

import * as esbuild from 'esbuild-wasm/lib/browser.js';
import { join } from 'path';
import { getProcessEnv, isBrowserLike } from './env';
import { captureError, HexclaveAssertionError, throwErr } from "./errors";
import { createGlobalAsync } from './globals';
import { ignoreUnhandledRejection, runAsynchronously } from './promises';
import { Result } from "./results";
import { traceSpan, withTraceSpan } from './telemetry';
type EsbuildBrowserRuntimeModule = Partial<typeof esbuild> & {
default?: typeof esbuild,
};
// The built ESM package can import esbuild-wasm's CJS browser entry as a
// default-only namespace under Node. Normalize once so both tsx source runs and
// built dist runs see the same API surface.
const esbuildModule: EsbuildBrowserRuntimeModule = esbuild;
function getEsbuildApi(module: EsbuildBrowserRuntimeModule): typeof esbuild {
return module.version == null && module.default != null
? module.default
: esbuild;
}
const esbuildApi = getEsbuildApi(esbuildModule);
// esbuild requires self property to be set, and it is not set by default in nodejs
(globalThis.self as any) ??= globalThis as any;
let esbuildInitializePromise: Promise<void> | null = null;
if (typeof process !== "undefined" && typeof process.exit === "function" && getProcessEnv("NODE_ENV") === 'development') {
// On development Node.js servers, initialize ESBuild as soon as the module is imported so we have to wait less on the first request
runAsynchronously(async () => {
try {
await initializeEsbuild();
} catch (e) {
captureError("initialize-esbuild-in-dev", e);
}
});
}
export function initializeEsbuild(): Promise<void> {
const esbuildWasmUrl = `https://unpkg.com/esbuild-wasm@${esbuildApi.version}/esbuild.wasm`;
if (esbuildInitializePromise == null) {
esbuildInitializePromise = withTraceSpan('initializeEsbuild', async () => {
try {
let initOptions;
if (isBrowserLike()) {
initOptions = {
wasmURL: esbuildWasmUrl,
};
} else {
const esbuildWasmModule = await createGlobalAsync('esbuildWasmModule', async () => {
const esbuildWasmResponse = await fetch(esbuildWasmUrl);
if (!esbuildWasmResponse.ok) {
throw new HexclaveAssertionError(`Failed to fetch esbuild.wasm: ${esbuildWasmResponse.status} ${esbuildWasmResponse.statusText}: ${await esbuildWasmResponse.text()}`);
}
const esbuildWasm = await esbuildWasmResponse.arrayBuffer();
const esbuildWasmArray = new Uint8Array(esbuildWasm);
if (esbuildWasmArray[0] !== 0x00 || esbuildWasmArray[1] !== 0x61 || esbuildWasmArray[2] !== 0x73 || esbuildWasmArray[3] !== 0x6d) {
throw new HexclaveAssertionError(`Invalid esbuild.wasm file: ${new TextDecoder().decode(esbuildWasmArray)}`);
}
return new WebAssembly.Module(esbuildWasm);
});
initOptions = {
wasmModule: esbuildWasmModule,
worker: false,
};
}
try {
await esbuildApi.initialize(initOptions);
} catch (e) {
if (e instanceof Error && e.message === 'Cannot call "initialize" more than once') {
// this happens especially in local development, just ignore
} else {
throw e;
}
}
} catch (e) {
esbuildInitializePromise = null;
throw new HexclaveAssertionError("Failed to initialize ESBuild", { cause: e });
}
})();
ignoreUnhandledRejection(esbuildInitializePromise);
}
return esbuildInitializePromise;
}
export async function bundleJavaScript(sourceFiles: Record<string, string> & { '/entry.js': string }, options: {
format?: 'iife' | 'esm' | 'cjs',
externalPackages?: Record<string, string>,
keepAsImports?: string[],
sourcemap?: false | 'inline',
allowHttpImports?: boolean,
} = {}): Promise<Result<string, string>> {
await initializeEsbuild();
const sourceFilesMap = new Map(Object.entries(sourceFiles));
const externalPackagesMap = new Map(Object.entries(options.externalPackages ?? {}));
const keepAsImports = options.keepAsImports ?? [];
const httpImportCache = new Map<string, { contents: string, loader: esbuild.Loader, resolveDir: string }>();
const extToLoader: Map<string, esbuild.Loader> = new Map([
['tsx', 'tsx'],
['ts', 'ts'],
['js', 'js'],
['jsx', 'jsx'],
['json', 'json'],
['css', 'css'],
]);
let result;
try {
result = await traceSpan('bundleJavaScript', async () => await esbuildApi.build({
entryPoints: ['/entry.js'],
bundle: true,
write: false,
format: options.format ?? 'iife',
platform: 'browser',
target: 'es2015',
jsx: 'automatic',
sourcemap: options.sourcemap ?? 'inline',
external: keepAsImports,
plugins: [
...options.allowHttpImports ? [{
name: "esm-sh-only",
setup(build: esbuild.PluginBuild) {
// Handle absolute URLs and relative imports from esm.sh modules.
build.onResolve({ filter: /.*/ }, (args) => {
// Only touch absolute http(s) specifiers or children of our own namespace
const isHttp = args.path.startsWith("http://") || args.path.startsWith("https://");
const fromEsmNs = args.namespace === "esm-sh";
if (!isHttp && !fromEsmNs) return null; // Let other plugins handle bare/relative/local
// Resolve relative URLs inside esm.sh-fetched modules
const url = new URL(args.path, fromEsmNs ? args.importer : undefined);
if (url.protocol !== "https:" || url.host !== "esm.sh") {
throw new Error(`Blocked non-esm.sh URL import: ${url.href}`);
}
return { path: url.href, namespace: "esm-sh" };
});
build.onLoad({ filter: /.*/, namespace: "esm-sh" }, async (args) => {
if (httpImportCache.has(args.path)) return httpImportCache.get(args.path)!;
const res = await fetch(args.path, { redirect: "follow" });
if (!res.ok) throw new Error(`Fetch ${res.status} ${res.statusText} for ${args.path}`);
const finalUrl = new URL(res.url);
// Defensive: follow shouldnt leave esm.sh, but re-check.
if (finalUrl.host !== "esm.sh") {
throw new Error(`Redirect escaped esm.sh: ${finalUrl.href}`);
}
const ct = (res.headers.get("content-type") || "").toLowerCase();
let loader: esbuild.Loader =
ct.includes("css") ? "css" :
ct.includes("json") ? "json" :
ct.includes("typescript") ? "ts" :
ct.includes("jsx") ? "jsx" :
ct.includes("tsx") ? "tsx" :
"js";
// Fallback by extension (esm.sh sometimes omits CT)
const p = finalUrl.pathname;
if (p.endsWith(".css")) loader = "css";
else if (p.endsWith(".json")) loader = "json";
else if (p.endsWith(".ts")) loader = "ts";
else if (p.endsWith(".tsx")) loader = "tsx";
else if (p.endsWith(".jsx")) loader = "jsx";
const contents = await res.text();
const result = {
contents,
loader,
// Ensures relative imports inside that module resolve against the files URL
resolveDir: new URL(".", finalUrl.href).toString(),
watchFiles: [finalUrl.href],
};
httpImportCache.set(args.path, result);
return result;
});
},
} as esbuild.Plugin] : [],
{
name: 'replace-packages-with-globals',
setup(build) {
build.onResolve({ filter: /.*/ }, args => {
// Skip packages that should remain external (not be shimmed)
if (keepAsImports.includes(args.path)) {
return undefined;
}
if (externalPackagesMap.has(args.path)) {
return { path: args.path, namespace: 'package-shim' };
}
return undefined;
});
build.onLoad({ filter: /.*/, namespace: 'package-shim' }, (args) => {
const contents = externalPackagesMap.get(args.path);
if (contents == null) throw new HexclaveAssertionError(`esbuild requested file ${args.path} that is not in the virtual file system`);
return { contents, loader: 'ts' };
});
},
},
{
name: 'virtual-fs',
setup(build) {
build.onResolve({ filter: /.*/ }, args => {
const absolutePath = join("/", args.path);
if (sourceFilesMap.has(absolutePath)) {
return { path: absolutePath, namespace: 'virtual' };
}
return undefined;
});
/* 2⃣ Load the module from the map */
build.onLoad({ filter: /.*/, namespace: 'virtual' }, args => {
const contents = sourceFilesMap.get(args.path);
if (contents == null) throw new HexclaveAssertionError(`esbuild requested file ${args.path} that is not in the virtual file system`);
const ext = args.path.split('.').pop() ?? '';
const loader = extToLoader.get(ext) ?? throwErr(`esbuild requested file ${args.path} with unknown extension ${ext}`);
return { contents, loader };
});
},
},
],
}));
} catch (e) {
if (e instanceof Error && e.message.startsWith("Build failed with ")) {
return Result.error(e.message);
}
throw e;
}
if (result.errors.length > 0) {
return Result.error(result.errors.map(e => e.text).join('\n'));
}
if (result.outputFiles.length > 0) {
return Result.ok(result.outputFiles[0].text);
}
return throwErr("No output generated??");
}