Skip to content

fix(studio-server): sub-composition previews build off the request thread within a deadline - #7857

Open
miguel-heygen wants to merge 4 commits into
mainfrom
fix/subcomposition-build-deadline
Open

miguel-heygen wants to merge 4 commits into
mainfrom
fix/subcomposition-build-deadline

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

What

GET /projects/:id/preview/comp/* now builds the sub-composition page on a worker thread, within a deadline (30 s by default). A page still building at the deadline is answered 503 with " took too long to build", and its worker is ended and replaced.

Why

A sub-composition page is the project's own HTML, and a shared or imported project can carry any page. The route built it on the server's request thread with no time limit, in three steps: the hf-id pin on the raw file, buildSubCompositionHtml, then the hf-id pass on the transformed page. A page big enough to take seconds held every other request to the server for that long. One that never finished held the server for good. The media-codec probes for the page's media had no limit either.

Related work

The root preview build already runs through the host's adapter.bundle, which a host can run off its main thread. This gives the sub-composition build the same separation inside studio-server itself, for every host.

How

  • helpers/subCompositionWorker.ts: the worker. A build job pins hf-ids on the raw file, as pinSubCompHfIds did, and runs buildSubCompositionHtml. An ids job pins hf-ids on the page the host transformed.
  • helpers/subCompositionBuild.ts: one long-lived worker, so builds queue one at a time. Its entry is found beside the module that starts it: the built .js in dist, else the .ts source through tsx, as shaderTransitionWorkerPool does.
    • runOffThread(job, deadline, compPath) refuses with SubCompositionDeadlineError once the deadline passes. The worker is then ended, and the builds queued behind the refused one go to a fresh worker.
    • A worker crash fails every build it held, since any of them may have caused it. They are not resent, so a page that crashes the worker cannot loop.
    • withinDeadline puts the same deadline on the media-codec probes. They are child processes, so nothing blocks; the probes themselves finish and fill the cache.
  • The route keeps one deadline for the whole request, across the build, the host transform, the second hf-id pass and the probes.
  • createStudioApi(adapter, { subCompositionDeadlineMs }) sets the deadline; tests use a short one.
  • tsup.config.ts in studio-server and the CLI emit subCompositionWorker.js beside the chunks that start it, as the CLI already does for shaderTransitionWorker.

Test plan

  • Unit tests added: src/routes/preview.compDeadline.test.ts
    • A page of 300,000 elements, with a 200 ms deadline, is answered 503, and the request thread never goes 200 ms without running a timer while it builds.
    • After a refused page, a small page still builds (the worker was replaced) and answers 200 with its content.
    • Fails without the fix: at the test-only commit the huge page answers 200 after building on the request thread, and the thread stalls for the whole build.
      (Linux run of both commits pending: the box is full; I add its result here.)
  • preview.compPath.test.ts (existing sub-composition route tests): run pending with the above.
  • Manual testing performed: not done. No CLI or Studio app was run against a real project.
  • Documentation updated: no user-facing docs name this route.
  • Comments follow CONTRIBUTING.md "Comments"

Not exercised: a packaged CLI build starting the worker from dist (the CI build compiles it; no hyperframes preview run was made), and Windows.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 267 (base branch 267), smooth 101 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes at b4fb520d. Moving the sub-composition build onto a worker under a deadline is the right shape, and the deadline/replace logic reads correctly. But the head doesn't build, and the worker can't start from source, so the new tests and the existing sub-composition route tests fail.

Blocking

  1. Typecheck/build fails. src/helpers/subCompositionBuild.ts:73: started.on("error", (error) => replace(started, error)). The error parameter is unknown, which gives TS2345: Argument of type 'unknown' is not assignable to parameter of type 'Error | undefined'. This fails Build, Typecheck, and every job that builds the workspace first (Test (studio/cli/runtime/support), CLI smoke, Producer, SDK). Fix: (error) => replace(started, error instanceof Error ? error : new Error(String(error))).

  2. The worker can't load from source, so every sub-composition preview returns 500 there. subCompositionWorker.ts imports ./subComposition.js. When the .ts entry runs in a Node worker, that relative .js doesn't map to the .ts file. Node 22.18+ strips types itself, so the worker file loads, but its import doesn't resolve. Adding --import tsx/esm to the worker's execArgv doesn't fix it either. I tried it directly on Node 22.22.3 and got ERR_MODULE_NOT_FOUND …/src/helpers/subComposition.js. The precedent, shaderTransitionWorker, only works because it has no relative imports.

    • Locally at this head: vitest run src/routes/preview.compDeadline.test.ts src/routes/preview.compPath.test.ts fails 4/4. Each request returns 500 with that error in stderr, including the two existing preview.compPath tests.
    • The same path affects any source-mode host, e.g. Studio dev loading studio-server from src.
    • Possible fixes: in source mode, start a small .mjs bootstrap that calls register() from tsx/esm/api and then import()s the .ts worker. Or keep the worker free of relative imports. Then re-run both test files.
  3. Comments check: createStudioApi.ts comment share rose (the new doc line on subCompositionDeadlineMs). Cut it or move it, per the ratchet's message.

Non-blocking

  • A deadline on a queued job kills the running one. When a job times out while another job is running (e.g. one request's ids job behind a different request's slow build), replace() ends the worker. The running build is resent and starts over from scratch. The comment "after a deadline the culprit is gone" doesn't hold in that case. It's bounded by each job's own deadline, so it isn't a correctness bug. Tracking the in-flight id, and only replacing when the timed-out job is the one running, would avoid it.
  • The remaining request-thread passes aren't bounded. After the worker returns, the route still runs recordPreviewReferences (an element walk), transformPreviewHtml, injectStudioPreviewAugmentations (which does a parseHTML at preview.ts:218) and the codec-map scan. All of these run synchronously on the request thread, over a page the worker took up to 30 s to build. withinDeadline can't interrupt synchronous work. Either a test with a page that builds just inside the deadline, or a note on why those passes are cheap relative to the build, would close this.
  • The test plan's "fails without the fix" run and the existing preview.compPath run are still pending in the description. Point 2 above is what they would have shown.

Need: real (a project's own page could hold the request thread with no bound) · Size: right-sized · Reuse: follows the existing worker-entry pattern, but that pattern doesn't carry over to a worker with relative imports · Simpler: fine · Structure: good split (route / pool / worker)
Risk: as posted, every sub-composition preview 500s in source mode and the workspace doesn't build. Once 1–3 are fixed, the risk is low, with the packaged-CLI dist worker path still unexercised.

— Rames

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants