fix(builders): copy traced runtime assets into the Vercel workflow function#2770
Draft
NathanColosimo wants to merge 6 commits into
Draft
fix(builders): copy traced runtime assets into the Vercel workflow function#2770NathanColosimo wants to merge 6 commits into
NathanColosimo wants to merge 6 commits into
Conversation
…nction Bundling inlines JavaScript but drops files that code loads from disk at runtime — Prisma query engines, native addons, data files. Trace the modules esbuild inlined into the steps bundle with @vercel/nft and copy the runtime assets they reference into flow.func so runtime lookups relative to the function root succeed. Fixes #1956
…imize lockfile churn
…s-scoped - Wrap tracing in try/catch so a trace failure degrades to the previous behavior (nothing copied) instead of failing builds - Only copy assets inside node_modules; app-directory files (which can include credentials) are never copied - Resolve absolute traced paths (cross-drive on Windows) directly
🦋 Changeset detectedLatest commit: 509780e The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
|
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1956
Problem
The Vercel Build Output API builder bundles all workflow/step code into
flow.func/index.mjs. Bundling inlines JavaScript, but drops files that code loads from disk at runtime.@prisma/clientis the reported case: its generated client resolves the native query engine (libquery_engine-*.so.node) andschema.prismanext to the generated client at runtime, so every step that touches the database crashes withPrismaClientInitializationErroron Vercel. The same applies to any native addon, WASM file, or data file a dependency reads at runtime.Fix
The steps-bundle esbuild metafile already lists every module that was inlined into the function bundle. After bundling, those modules are traced with
@vercel/nftat their original on-disk locations (where relative asset references still resolve), and the runtime assets they reference are copied intoflow.func:asset(static analysis of fs reads) orsharedlib, or when it's a.nodenative addon (nft traces those as plaindependency, and itssharedlibglobs are platform-specific — a.dylib.nodePrisma engine traced on Linux gets no tag).node_modulesare copied. They keep their path below the innermostnode_modulesdirectory, which flattens pnpm/store layouts (node_modules/.pnpm/<pkg>/node_modules/.prisma/client/engine.node→node_modules/.prisma/client/engine.node). That's the flat layout runtime lookups probe, resolved against the function root — which isprocess.cwd()at runtime.package.jsonis copied alongside each asset so runtime resolution of the copied files keeps working (e.g. a native package whosemainpoints at a.nodebinary).node_modulesare deliberately never copied (a warning is logged), so app-level credentials (service-account.json,.env, keys) can't ride into the deployed function. Credential-suffixed files are additionally denylisted as defense-in-depth.Because the trace entries come from the metafile, everything esbuild already resolved (tsconfig path aliases, package export conditions, the SWC plugin's output) is handled for free — no parallel resolution logic. The metafile is skipped in watch mode, so dev rebuilds don't pay for it.
@vercel/nftis pinned to 0.30.4 (last version supporting Node 18, which the workspace still advertises).This runs for
VercelBuildOutputAPIBuilderand everything that extends it (CLI--target vercel-build-output-api, Nitro/Nuxt, Astro, SvelteKit Vercel builders).Tests
mainwith exactly the issue's failure mode (engine missing fromflow.func) and pass with the fix:node_moduleslayout (the Prisma engine binary not bundled with workflow steps on Vercel #1956 scenario), asserting engine + schema + owningpackage.jsonare copiedlibquery_engine-darwin.dylib.nodeon purpose: it matches no nftsharedlibglob on any platform, so the test also locks in the.node-addon path.@workflow/builderssuite: 422 tests green. Typecheck + build green.workbench/exampleviaworkflow build --target vercel-build-output-apiwith the same fixture; the engine landed inflow.func/node_modules/.fake-prisma/client/and was readable through Prisma's cwd-based lookup. Across the whole example app the trace copied exactly two assets (the fixture engine andcbor-x/dist/node.cjs), i.e. no output bloat.Known limitations (follow-up territory)
__dirname/import.meta.url-relative reads inside bundled code resolve against the function root and aren't remapped (Prisma is unaffected — it probescwd()/node_modules/.prisma/client).readFileSync(join(process.cwd(), 'data/config.json'))) are not copied; onlynode_modulesassets are. A build warning points at each skipped file.node_modulespath, the first in trace order wins (a warning is logged on conflict).Review
Reviewed with
/simplify(3-agent reuse/quality/efficiency pass) and 4 rounds of autoreview (Codexgpt-5.5:xhigh+ Claude Opus4.8:xhighpanel); all accepted findings addressed.