Skip to content

feat: introduce v8 locker in our codebase #549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Jun 18, 2025
Merged

feat: introduce v8 locker in our codebase #549

merged 22 commits into from
Jun 18, 2025

Conversation

nyannyacha
Copy link
Contributor

@nyannyacha nyannyacha commented May 30, 2025

What kind of change does this PR introduce?

Feature, Enhancement

Description

Introducing the v8 Locker API to our code base.

Deno prefers to have only one runtime in a single thread, but it is possible to have multiple runtimes in a single thread as long as thread-local variables are managed properly.

edge-runtime may need to manage hundreds (or even thousands) of runtimes in a single process, so instead of matching threads and runtimes one-to-one, we chose a model where a tokio thread can have multiple runtimes.

While this model avoided the overhead of spawning extra threads by having only a small number of threads, it also meant that there was only a fixed number of runtimes that any particular thread could process at any given time, so the slower any runtime processed, the slower the other runtimes tied to the same thread would process.
To mitigate this issue, we can temporarily separate the runtime that is reached particular stage, which is expected to take up most of the lifecycle, such as the bootstrapping stage, from the tokio thread and process it in a separate thread pool so that the processing time of the tokio thread does not slow down.

The problem is that in order to enable the runtime to move between multiple threads, we need the Locker API from v8.
Deno had previously attempted to introduce this API into their library, but ultimately gave up.

We ended up forking their library and introducing the API they had given up.

What's changed (or fixed)

  • Changed to use the Locker API to transfer the runtime to another thread to handle if the runtime is at a stage where it can become a bottleneck (e.g., bootstrapping).
  • Fixes an issue that allowed certain workers to be pulled and reused in the main script while being transitioned to the retired state.
    • A new error called WorkerAlreadyRetired has been introduced. If you try to reuse a retired worker (i.e., call worker.fetch), the error will be thrown. (Please refer to the updated example/main/index.ts)
  • Fixes a bug where the keys in the vfs used during the eszip bundling process were not properly sorted, causing binary searches to be performed incorrectly when subsequently loading eszip and searching for files or directories in the vfs.
  • Added various load tests.
  • The introduction of the Locker API allows some synchronous file APIs to be used in a top-level context, which was not allowed before.
    • For a list of allowed synchronous file APIs, see here.
    • Example
    import meow from "meowmeow";
    
    Deno.readFileSync("..."); // ✅ (1)
    Deno.serve(() => {
      // Some callback context (polled in the event loop of the main tokio thread). (4)
      Deno.readFileSync("..."); // 💣 ERROR! Deno.readFileSync is blocklisted
    });
    
    // ...snip... (2)
    
    // All top-level statements have been evaluated, so the Sync API is unavailable from now on... (3)

Closes FUNC-183, FUNC-215

nyannyacha added 11 commits May 30, 2025 18:38
DESCRIPTION:
vfs uses binary search internally, so it must be properly sorted.
However, it is inappropriate to use IndexMap::(...)::collect() and
expect the elements to be sorted.
DESCRIPTION:
When the main worker's script relays an HTTP request to
the user worker, it may already be in a retired state.
@nyannyacha nyannyacha marked this pull request as ready for review June 10, 2025 01:45
@nyannyacha nyannyacha changed the title [WIP] feat: introduce v8 locker in our codebase feat: introduce v8 locker in our codebase Jun 18, 2025
@nyannyacha nyannyacha requested a review from laktek June 18, 2025 01:53
@laktek
Copy link
Contributor

laktek commented Jun 18, 2025

Given the nature of the change, let's merge and conduct some additional stress testing within internal staging environment.

@nyannyacha nyannyacha merged commit c56fc63 into develop Jun 18, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Edge Runtime fails to resolve npm packages from registries mounted under a subpath (e.g. /some-path/)
2 participants