src: add web locks api

[IlyasShabi]

This PR implements the Web Locks API, Locks are used to coordinate access to shared resources across multiple threads.

This implementation is based on previous work in #22719 and #36502, but takes a C++ native approach for better performance and reliability, this solution uses a singleton LockManager with thread-safe data structures to coordinate locks across all workers.

• Support exclusive and shared modes
• Support steal option
• Support ifAvailable option
• Support signal option
• Documentation
• WPT tests
• Add missing query.https.any.js tests as unit tests
• Add basic tests

Closes: #22702
Refs: https://w3c.github.io/web-locks

[nodejs-github-bot]

Review requested:

• @nodejs/gyp
• @nodejs/startup
• @nodejs/web-standards

[codecov]

Codecov Report

Attention: Patch coverage is 79.88636% with 177 lines in your changes missing coverage. Please review.

Project coverage is 90.01%. Comparing base (09b4c57) to head (66089c5).

Files with missing lines	Patch %	Lines
src/node_locks.cc	70.20%	95 Missing and 68 partials ⚠️
lib/internal/locks.js	95.22%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #58666      +/-   ##
==========================================
- Coverage   90.07%   90.01%   -0.07%     
==========================================
  Files         641      644       +3     
  Lines      188998   189878     +880     
  Branches    37069    37245     +176     
==========================================
+ Hits       170246   170917     +671     
- Misses      11462    11599     +137     
- Partials     7290     7362      +72     

Files with missing lines	Coverage Δ
lib/internal/navigator.js	99.37% <100.00%> (+0.04%)	⬆️
lib/worker_threads.js	100.00% <100.00%> (ø)
src/node_binding.cc	82.67% <ø> (ø)
src/node_external_reference.h	100.00% <ø> (ø)
src/node_locks.h	100.00% <100.00%> (ø)
lib/internal/locks.js	95.22% <95.22%> (ø)
src/node_locks.cc	70.20% <70.20%> (ø)

... and 35 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mcollina]

lgtm

[Ethan-Arrowood]

this lgtm. I looked through the WPT and other tests and they look all good. I reviewed the JS api to make sure it aligns with expectations and analyzed its source. All is good for a first implementation. I did a cursory review of the C++ part as I'm less experienced there, but in general it looks okay too. Nice work!

[panva]

Adding semver-major PRs that contain breaking changes and should be released in the next major version. as this adds new globals (Lock, LockManager)

[github-actions]

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @anonrig.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/67658/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/67670/

[jasnell]

Just putting the red x on this since CI passed and it has sufficient sign off from others to land but there are still a few outstanding issues to resolve. I want to make sure it doesn't end up getting landed by the commit-queue while some comments are still pending.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/67744/

[jasnell]

These's aren't correct here. You really should not be holding a v8::Global<T> inside a v8::External, then deleting it like this. It's fine to use indirection through another type... e.g. having the External hold an instance of a struct that is holding the Global<T>.

[IlyasShabi]

I did experiment with a wrapper that let the callback own a
std::unique_ptr, so the Global<…> would be reset automatically, but that changed the timing of the reset and introduced a race that broke the WPT and hanging tests sometimes.

I think that the current pattern is also valid since the Global is reset and freed immediately in the callback and the raw pointer’s ownership is transferred with release(). Wdyt ?

[jasnell]

I ought to be able to take another look this afternoon. Just catching up after two week of being out of the office ;-) ... just mentioning because I didn't want you to feel I was ignoring your pings on this ;-)

[IlyasShabi]

Take your time, thanks for your help

[IlyasShabi]

@jasnell Did you manage to take another look on this?

[addaleax]

Yeah, this pattern is indeed very odd. Here's a diff that simplifies things a lot without appearing to break anything:

@@ -132,24 +130,15 @@ static void OnLockCallbackRejected(const FunctionCallbackInfo<Value>& info) {
 // Called when the promise returned from the user's callback resolves
 static void OnIfAvailableFulfill(const FunctionCallbackInfo<Value>& info) {
   HandleScope handle_scope(info.GetIsolate());
-  auto* holder = static_cast<Global<Promise::Resolver>*>(
-      info.Data().As<External>()->Value());
-  USE(holder->Get(info.GetIsolate())
+  USE(info.Data().As<Promise::Resolver>()
           ->Resolve(info.GetIsolate()->GetCurrentContext(), info[0]));
-  holder->Reset();
-  delete holder;
 }
 
 // Called when the promise returned from the user's callback rejects
 static void OnIfAvailableReject(const FunctionCallbackInfo<Value>& info) {
   HandleScope handle_scope(info.GetIsolate());
-  auto* holder = static_cast<Global<Promise::Resolver>*>(
-      info.Data().As<External>()->Value());
-  USE(holder->Get(info.GetIsolate())
+  USE(info.Data().As<Promise::Resolver>()
           ->Reject(info.GetIsolate()->GetCurrentContext(), info[0]));
-  holder->Reset();
-
-  delete holder;
 }
 
 void LockManager::CleanupStolenLocks(Environment* env) {
@@ -326,28 +315,17 @@ void LockManager::ProcessQueue(Environment* env) {
       if (callback_result->IsPromise()) {
         Local<Promise> p = callback_result.As<Promise>();
 
-        // The resolver survives until the promise settles and is freed
-        // automatically.
-        auto resolve_holder = std::make_unique<Global<Promise::Resolver>>(
-            isolate, if_available_request->released_promise());
-        auto reject_holder = std::make_unique<Global<Promise::Resolver>>(
-            isolate, if_available_request->released_promise());
-
         Local<Function> on_fulfilled;
         Local<Function> on_rejected;
         CHECK(Function::New(context,
                             OnIfAvailableFulfill,
-                            External::New(isolate, resolve_holder.get()))
+                            if_available_request->released_promise())
                   .ToLocal(&on_fulfilled));
         CHECK(Function::New(context,
                             OnIfAvailableReject,
-                            External::New(isolate, reject_holder.get()))
+                            if_available_request->released_promise())
                   .ToLocal(&on_rejected));
 
-        // Transfer ownership to the callbacks.
-        resolve_holder.release();
-        reject_holder.release();
-
         {
           TryCatchScope try_catch_scope(env);
           if (p->Then(context, on_fulfilled, on_rejected).IsEmpty()) {

Generally speaking, we:

• Strongly want to avoid using External, because it is an "unsafe" feature of the V8 API in the sense that it associates a C++ object with a JS object without leaving V8's Garbage Collector, C++ memory management or diagnostic tooling any insight into what's happening (and so an External can easily end up accidentally containing a dangling pointer, or alternatively the C++ object not freed properly when the External is garbage-collected).
• Avoid using Globals, because those do not properly represent references-relationships between objects (disclaimer: I don't know how the current work on C++ garbage collector integration changes this). If a Global pointing at a JS object B is contained in a C++ object that is attached to some JS object A, the garbage collector cannot see this reference, and it treats the Global as a separate GC root (or a weak reference), neither of which correctly represents the typical intended reference mechanism.

In this case, we should be able to just associate JS data directly with the JS function object in question. If this isn't possible, it's generally advisable to create a BaseObject with internal fields (in the V8 sense of the word) instead, which solves these issues.

I think the latter (inherit from BaseObject) is what we'd also want to do for LockHolder, instead of creating Externals containing pointers to LockHolder instances.

[jasnell]

I still think there may be at least a few issues here relating to how you're using v8::Global and v8::External. I'd really like to see if we could get @addaleax to take a look.

[addaleax]

Thank for the pings, in particular @BridgeAR for making me aware of this PR over Slack, where messages generally reach me a bit better 🙂

This PR looks great and I am excited to see this feature finally come to Node.js, I think @jasnell had some good comments on the memory management aspects here that I think should be addressed before this PR is merged

[addaleax]

Fwiw, this should be the change that @jasnell suggested in the resolved comment, still feels a bit cleaner here to me but obviously not a big deal either way

Suggested change

	auto* lock_holder =
	static_cast<LockHolder*>(info.Data().As<External>()->Value());
	std::shared_ptr<Lock> lock = lock_holder->lock();
	delete lock_holder;
	std::unique_ptr<LockHolder> lock_holder {
	static_cast<LockHolder*>(info.Data().As<External>()->Value()) };
	std::shared_ptr<Lock> lock = lock_holder->lock();

(similar pattern would apply below)

[addaleax]

Yeah, this pattern is indeed very odd. Here's a diff that simplifies things a lot without appearing to break anything:

@@ -132,24 +130,15 @@ static void OnLockCallbackRejected(const FunctionCallbackInfo<Value>& info) {
 // Called when the promise returned from the user's callback resolves
 static void OnIfAvailableFulfill(const FunctionCallbackInfo<Value>& info) {
   HandleScope handle_scope(info.GetIsolate());
-  auto* holder = static_cast<Global<Promise::Resolver>*>(
-      info.Data().As<External>()->Value());
-  USE(holder->Get(info.GetIsolate())
+  USE(info.Data().As<Promise::Resolver>()
           ->Resolve(info.GetIsolate()->GetCurrentContext(), info[0]));
-  holder->Reset();
-  delete holder;
 }
 
 // Called when the promise returned from the user's callback rejects
 static void OnIfAvailableReject(const FunctionCallbackInfo<Value>& info) {
   HandleScope handle_scope(info.GetIsolate());
-  auto* holder = static_cast<Global<Promise::Resolver>*>(
-      info.Data().As<External>()->Value());
-  USE(holder->Get(info.GetIsolate())
+  USE(info.Data().As<Promise::Resolver>()
           ->Reject(info.GetIsolate()->GetCurrentContext(), info[0]));
-  holder->Reset();
-
-  delete holder;
 }
 
 void LockManager::CleanupStolenLocks(Environment* env) {
@@ -326,28 +315,17 @@ void LockManager::ProcessQueue(Environment* env) {
       if (callback_result->IsPromise()) {
         Local<Promise> p = callback_result.As<Promise>();
 
-        // The resolver survives until the promise settles and is freed
-        // automatically.
-        auto resolve_holder = std::make_unique<Global<Promise::Resolver>>(
-            isolate, if_available_request->released_promise());
-        auto reject_holder = std::make_unique<Global<Promise::Resolver>>(
-            isolate, if_available_request->released_promise());
-
         Local<Function> on_fulfilled;
         Local<Function> on_rejected;
         CHECK(Function::New(context,
                             OnIfAvailableFulfill,
-                            External::New(isolate, resolve_holder.get()))
+                            if_available_request->released_promise())
                   .ToLocal(&on_fulfilled));
         CHECK(Function::New(context,
                             OnIfAvailableReject,
-                            External::New(isolate, reject_holder.get()))
+                            if_available_request->released_promise())
                   .ToLocal(&on_rejected));
 
-        // Transfer ownership to the callbacks.
-        resolve_holder.release();
-        reject_holder.release();
-
         {
           TryCatchScope try_catch_scope(env);
           if (p->Then(context, on_fulfilled, on_rejected).IsEmpty()) {

Generally speaking, we:

• Strongly want to avoid using External, because it is an "unsafe" feature of the V8 API in the sense that it associates a C++ object with a JS object without leaving V8's Garbage Collector, C++ memory management or diagnostic tooling any insight into what's happening (and so an External can easily end up accidentally containing a dangling pointer, or alternatively the C++ object not freed properly when the External is garbage-collected).
• Avoid using Globals, because those do not properly represent references-relationships between objects (disclaimer: I don't know how the current work on C++ garbage collector integration changes this). If a Global pointing at a JS object B is contained in a C++ object that is attached to some JS object A, the garbage collector cannot see this reference, and it treats the Global as a separate GC root (or a weak reference), neither of which correctly represents the typical intended reference mechanism.

In this case, we should be able to just associate JS data directly with the JS function object in question. If this isn't possible, it's generally advisable to create a BaseObject with internal fields (in the V8 sense of the word) instead, which solves these issues.

I think the latter (inherit from BaseObject) is what we'd also want to do for LockHolder, instead of creating Externals containing pointers to LockHolder instances.