diff --git a/doc/contributing/adding-v8-fast-api.md b/doc/contributing/adding-v8-fast-api.md index e26bc841121e66..f805c854e048f4 100644 --- a/doc/contributing/adding-v8-fast-api.md +++ b/doc/contributing/adding-v8-fast-api.md @@ -1,84 +1,407 @@ -# Adding V8 Fast API - -Node.js uses [V8](https://v8.dev/) as its JavaScript engine. -Embedding functions implemented in C++ incur a high overhead, so V8 -provides an API to implement native functions which may be invoked directly -from JIT-ed code. These functions also come with additional constraints, -for example, they may not trigger garbage collection. - -## Limitations - -* Fast API functions may not trigger garbage collection. This means by proxy - that JavaScript execution and heap allocation are also forbidden, including - `v8::Array::Get()` or `v8::Number::New()`. -* Throwing errors is not available from within a fast API call, but can be done - through the fallback to the slow API. -* Not all parameter and return types are supported in fast API calls. - For a full list, please look into - [`v8-fast-api-calls.h`](../../deps/v8/include/v8-fast-api-calls.h). - -## Requirements - -* Any function passed to `CFunction::Make`, including fast API function - declarations, should have their signature registered in - [`node_external_reference.h`](../../src/node_external_reference.h) file. - Although, it would not start failing or crashing until the function ends up - in a snapshot (either the built-in or a user-land one). Please refer to the - [binding functions documentation](../../src/README.md#binding-functions) for more - information. -* Fast API functions must be tested following the example in - [Test with Fast API path](#test-with-fast-api-path). -* The fast callback must be idempotent up to the point where error and fallback - conditions are checked, because otherwise executing the slow callback might - produce visible side effects twice. -* If the receiver is used in the callback, it must be passed as a second argument, - leaving the first one unused, to prevent the JS land from accidentally omitting the receiver when - invoking the fast API method. +# Adding V8 Fast API callbacks + +Node.js uses [V8](https://v8.dev/) as its JavaScript engine. Embedding +functions implemented in C++ incurs a high overhead, so V8 provides an API to +implement native C++ functions which may be invoked directly from JIT-ed code. + +Early iterations of the Fast API imposed significant constraints on these +functions, such as not allowing re-entry into JavaScript execution and not +throwing errors directly from fast calls. As of V8 12.6, these constraints no +longer exist; however, a function whose execution cost is far higher than its +calling cost is unlikely to benefit from having a "fast" variant, so some +judgement is required when considering whether or not to add a Fast API +callback. + +## Basics + +A Fast API callback must correspond to a conventional ("slow") implementation +of the same callback. Compare the two conventions: + +```cpp +// Conventional ("slow") implementation +void IsEven(const v8::FunctionCallbackInfo& args) { + Environment* env = Environment::GetCurrent(args); + if (!args[0]->IsInt32()) { + return THROW_ERR_INVALID_ARG_TYPE(env, "argument must be an integer"); + } - ```cpp - // Instead of invoking the method as `receiver.internalModuleStat(input)`, the JS land should - // invoke it as `internalModuleStat(binding, input)` to make sure the binding is available to - // the native land. - static int32_t FastInternalModuleStat( - Local unused, - Local recv, - const FastOneByteString& input, - FastApiCallbackOptions& options) { - Environment* env = Environment::GetCurrent(recv->GetCreationContextChecked()); - // More code + int32_t n = args[0]->Int32Value(env->context()).FromJust(); + bool result = n % 2 == 0; + args.GetReturnValue().Set(result); +} + +// Fast implementation +bool FastIsEven(v8::Local receiver, + const int32_t n) { + return n % 2 == 0; +} +static v8::CFunction fast_is_even(v8::CFunction::Make(FastIsEven)); +``` + +The main differences between the two call conventions are: + +* A conventional call passes its arguments as `v8::Value` objects, via a + `v8::FunctionCallbackInfo` object. A Fast API call passes its arguments + directly to the C++ function, as native C++ types where possible. +* A conventional call passes its return value via a `v8::ReturnValue` object, + accessible via the `v8::FunctionCallbackInfo` object. A Fast API call returns + its value directly from the C++ function, as a native C++ type. +* A conventional call can pass any number of arguments of any type, which must + be validated within the implementation. A Fast API callback will only ever be + called in compliance with its function signature, so the `FastIsEven` example + above will only ever be called with a single argument of type `int32_t`. Any + calls from JavaScript whose arguments do not correspond to a fast callback + signature will be directed to the slow path by V8, even if the function is + optimized. +* The fast callback cannot be bound directly. It must first be used to build a + `v8::CFunction` handle, which is passed alongside the conventional callback + when binding the function. + +## Argument and return types + +The following are valid argument types in Fast API callback signatures: + +* `bool` +* `int32_t` +* `uint32_t` +* `int64_t` +* `uint64_t` +* `float` +* `double` +* `v8::Local` (analogous to `any`) +* `v8::FastOneByteString&` (analogous to `string`, but _only_ allows sequential + one-byte strings, which is often not useful) + + + +The list of valid return types is similar: + +* `void` +* `bool` +* `int32_t` +* `uint32_t` +* `int64_t` +* `uint64_t` +* `float` +* `double` + + + +### Prepending a `receiver` argument + +V8 will always pass the "receiver" (the `this` value of the JavaScript function +call) in the first argument position. The arguments to the JavaScript function +call are then passed from the second position onwards. + +```cpp +// Let's say that this function was bound as a method on some object, +// such that it would be called in JavaScript as `object.hasProperty(foo)`. +bool FastHasProperty(v8::Local receiver, + v8::Local property, + v8::FastApiCallbackOptions& options) { + v8::Isolate* isolate = options.isolate; + + if (!receiver->IsObject()) { + // invalid `this` value; throw some kind of error here } - ``` -## Fallback to slow path + bool result; + if (!receiver.As()->Has(isolate->GetCurrentContext(), + property).To(&result)) { + // error pending in V8, value is ignored + return false; + } -Fast API supports fallback to slow path for when it is desirable to do so, -for example, when throwing a custom error or executing JavaScript code is -needed. The fallback mechanism can be enabled and changed from the C++ -implementation of the fast API function declaration. + return result; +} +``` -Passing `true` to the `fallback` option will force V8 to run the slow path -with the same arguments. +Even if your function binding does not need access to the receiver, you must +still prepend it to your function arguments. -In V8, the options fallback is defined as `FastApiCallbackOptions` inside -[`v8-fast-api-calls.h`](../../deps/v8/include/v8-fast-api-calls.h). +```cpp +bool FastIsObject(v8::Local receiver, // unused + v8::Local value) { + return value->IsObject(); +} +``` -* C++ land +### Appending an `options` argument (optional) + +Fast callbacks may add an optional final function argument of type +`v8::FastApiCallbackOptions&`. This is required if the callback interacts with +the isolate in any way: see +[Stack-allocated objects and garbage collection](#stack-allocated-objects-and-garbage-collection) +and [Handling errors](#handling-errors). + +```cpp +void FastThrowExample(v8::Local receiver, + const int32_t n, + v8::FastApiCallbackOptions& options) { + if (IsEvilNumber(n)) { + v8::HandleScope handle_scope(options.isolate); + THROW_ERR_INVALID_ARG_VALUE(options.isolate, "Begone, foul spirit!"); + } +} +``` - Example of a conditional fast path on C++ +## Registering a Fast API callback - ```cpp - // Anywhere in the execution flow, you can set fallback and stop the execution. - static double divide(const int32_t a, - const int32_t b, - v8::FastApiCallbackOptions& options) { - if (b == 0) { - options.fallback = true; - return 0; - } else { - return a / b; - } +Compare registering a conventional API binding: + +```cpp +void Initialize(Local target, + Local unused, + Local context, + void* priv) { + Environment* env = Environment::GetCurrent(context); + SetMethodNoSideEffect(context, target, "isEven", IsEven); +} +``` + +with registering an API binding with a fast callback: + +```cpp +void Initialize(Local target, + Local unused, + Local context, + void* priv) { + Environment* env = Environment::GetCurrent(context); + SetFastMethodNoSideEffect(context, + target, + "isEven", + SlowIsEven, + &fast_is_even); +} +``` + +The Fast API equivalents of the method binding functions take an additional +parameter, which specifies the fast callback(s). + +In the majority of cases, there will only be a single fast callback, and the +additional parameter should be a pointer to the `v8::CFunction` object +constructed by the call to `CFunction::Make`. + +In rare cases, there may be more than one fast callback, _eg._ if the function +accepts optional arguments. In this case, the additional parameter should be a +reference to an array of `v8::CFunction` objects, which is used to initialize a +`v8::MemorySpan`: + +```cpp +int32_t FastFuncWithoutArg(v8::Local receiver) { + return -1; +} +int32_t FastFuncWithArg(v8::Local receiver, + const v8::FastOneByteString& s) { + return s.length; +} +static CFunction fast_func_callbacks[] = {CFunction::Make(FastFuncWithoutArg), + CFunction::Make(FastFuncWithArg)}; + +void Initialize(Local target, + Local unused, + Local context, + void* priv) { + Environment* env = Environment::GetCurrent(context); + SetFastMethodNoSideEffect(context, + target, + "func", + SlowFunc, + fast_func_callbacks); +} +``` + +In addition, all method bindings should be registered with the external +reference registry. This is done by passing both the conventional callback +pointer and the `v8::CFunction` handle to `registry->Register`. + +```cpp +void RegisterExternalReferences(ExternalReferenceRegistry* registry) { + registry->Register(SlowIsEven); + registry->Register(fast_is_even); +} +``` + +Omitting this step can lead to fatal exceptions if the callback ends up in a +snapshot (either the built-in snapshot, or a user-land one). Refer to the +[binding functions documentation](../../src/README.md#registering-binding-functions-used-in-bootstrap) +for more information. + +## Type checking + +A callback argument that is a "primitive" C++ type (for example, `int32_t`) +does not require type checks, as V8 will only ever invoke the fast callback if +the argument in the JavaScript function call matches the corresponding argument +type in the fast callback signature. + +Non-primitive arguments (such as TypedArrays) are passed to Fast API callbacks +as `v8::Local`. However, registering a fast callback with this +argument type signals to the V8 engine that it can invoke the fast callback +with _any value_ as that argument. + +If using arguments of type `v8::Local`, then it is the +implementation's responsibility to ensure that the arguments are validated +before casting or otherwise consuming them. This can either take place within +the C++ callbacks themselves, or within a JavaScript wrapper function that +performs any necessary validation before calling the bound function. + +## Stack-allocated objects and garbage collection + +The Fast API now allows access to the isolate, and allows allocation of +`v8::Local` handles on the stack. + +A fast callback intending to make use of this functionality should accept a +final argument of type `v8::FastApiCallbackOptions&`. V8 will pass the isolate +pointer in `options.isolate`. + +If a fast callback creates any `v8::Local` handles within the fast callback, +then it must first initialize a new `v8::HandleScope` to ensure that the +handles are correctly scoped and garbage-collected. + +```cpp +bool FastIsIterable(v8::Local receiver, + v8::Local argument, + v8::FastApiCallbackOptions& options) { + if (!argument->IsObject()) { + return false; } - ``` + + // In order to create any Local handles, we first need a HandleScope + v8::HandleScope HandleScope(options.isolate); + + v8::Local object = argument.As(); + v8::Local value; + if (!object->Get(options.isolate->GetCurrentContext(), + v8::Symbol::GetIterator(options.isolate)).ToLocal(&value)) { + return false; + } + return value->IsFunction(); +} +``` + +The same applies if the fast callback calls other functions which themselves +create `v8::Local` handles, unless those functions create their own +`v8::HandleScope`. In general, if the fast callback interacts with +`v8::Local` handles within the body of the callback, it likely needs a handle +scope. + +## Debug tracking of Fast API callbacks + +In order to allow the test suite to track when a function call uses the Fast +API path, add the `TRACK_V8_FAST_API_CALL` macro to your fast callback. + +```cpp +bool FastIsEven(v8::Local receiver, + const int32_t n) { + TRACK_V8_FAST_API_CALL("util.isEven"); + return n % 2 == 0; +} +``` + +The tracking key must be unique, and should be of the form: + +` "." [ "." ]` + +The above example assumes that the fast callback is bound to the `isEven` +method of the `util` module binding. To track specific subpaths within the +callback, use a key with a subpath specifier, like `"util.isEven.error"`. + +These tracking events can be observed in debug mode, and are used to test that +the fast path is being correctly invoked. See +[Testing Fast API callbacks](#testing-fast-api-callbacks) for details. + +## Handling errors + +It is now possible to throw errors from within the Fast API. + +Any fast callback that might potentially need to throw an error back to the +JavaScript environment should accept a final `options` argument of type +`v8::FastApiCallbackOptions&`. V8 will pass the isolate pointer in +`options.isolate`. + +The callback should then throw a JavaScript error in the standard fashion. It +also needs to return a dummy value, to satisfy the function signature. + +As above, initializing a `v8::HandleScope` is mandatory before any operations +which create local handles. + +```cpp +static double FastDivide(v8::Local receiver, + const int32_t a, + const int32_t b, + v8::FastApiCallbackOptions& options) { + if (b == 0) { + TRACK_V8_FAST_API_CALL("math.divide.error"); + v8::HandleScope handle_scope(options.isolate); + THROW_ERR_INVALID_ARG_VALUE(options.isolate, + "cannot divide by zero"); + return 0; // dummy value, ignored by V8 + } + + TRACK_V8_FAST_API_CALL("math.divide.ok"); + return a / b; +} +``` + +## Testing Fast API callbacks + +To force V8 to use a Fast API path in testing, use V8 natives to force +optimization of the JavaScript function that calls the fast target. If +importing the binding directly, you will need to wrap the call within a +JavaScript function first. + +```js +// Flags: --allow-natives-syntax --expose-internals --no-warnings + +const common = require('../common'); +const assert = require('assert'); + +const { internalBinding } = require('internal/test/binding'); +const { isEven } = internalBinding('...'); + +function testFastAPICall() { + assert.strictEqual(isEven(0), true); +} + +// The first V8 directive prepares the wrapper function for optimization. +eval('%PrepareFunctionForOptimization(testFastAPICall)'); +// This call will use the slow path. +testFastAPICall(); + +// The second V8 directive will trigger optimization. +eval('%OptimizeFunctionOnNextCall(testFastAPICall)'); +// This call will use the fast path. +testFastAPICall(); +``` + +In debug builds, it is possible to observe +[`TRACK_V8_FAST_API_CALL`](#debug-tracking-of-fast-api-callbacks) events using +the`getV8FastApiCallCount` function, to verify that the fast path is being +correctly invoked. All fast callbacks should be tested in this way. + +```js +function testFastAPICalls() { + assert.strictEqual(isEven(1), false); + assert.strictEqual(isEven(2), true); +} + +eval('%PrepareFunctionForOptimization(testFastAPICalls)'); +testFastAPICalls(); +eval('%OptimizeFunctionOnNextCall(testFastAPICalls)'); +testFastAPICalls(); + +if (common.isDebug) { + const { getV8FastApiCallCount } = internalBinding('debug'); + assert.strictEqual(getV8FastApiCallCount('util.isEven'), 2); +} +``` ## Example @@ -99,48 +422,60 @@ A typical function that communicates between JavaScript and C++ is as follows. namespace node { namespace custom_namespace { + using v8::FastApiCallbackOptions; + using v8::FunctionCallbackInfo; + using v8::HandleScope; + using v8::Int32; + using v8::Number; + using v8::Value; + static void SlowDivide(const FunctionCallbackInfo& args) { Environment* env = Environment::GetCurrent(args); - CHECK_GE(args.Length(), 2); - CHECK(args[0]->IsInt32()); - CHECK(args[1]->IsInt32()); - auto a = args[0].As(); - auto b = args[1].As(); + if (!args[0]->IsInt32() || !args[1]->IsInt32()) { + return THROW_ERR_INVALID_ARG_TYPE(env, "operands must be integers"); + } + auto a = args[0].As(); + auto b = args[1].As(); if (b->Value() == 0) { - return node::THROW_ERR_INVALID_STATE(env, "Error"); + return THROW_ERR_INVALID_ARG_VALUE(env, "cannot divide by zero"); } double result = a->Value() / b->Value(); - args.GetReturnValue().Set(v8::Number::New(env->isolate(), result)); + args.GetReturnValue().Set(Number::New(env->isolate(), result)); } - static double FastDivide(const int32_t a, + static double FastDivide(v8::Local receiver, + const int32_t a, const int32_t b, - v8::FastApiCallbackOptions& options) { + FastApiCallbackOptions& options) { if (b == 0) { TRACK_V8_FAST_API_CALL("custom_namespace.divide.error"); - options.fallback = true; + HandleScope handle_scope(options.isolate); + THROW_ERR_INVALID_ARG_VALUE(options.isolate, "cannot divide by zero"); return 0; - } else { - TRACK_V8_FAST_API_CALL("custom_namespace.divide.ok"); - return a / b; } + + TRACK_V8_FAST_API_CALL("custom_namespace.divide.ok"); + return a / b; } - CFunction fast_divide_(CFunction::Make(FastDivide)); + static CFunction fast_divide(CFunction::Make(FastDivide)); static void Initialize(Local target, Local unused, Local context, void* priv) { - SetFastMethod(context, target, "divide", SlowDivide, &fast_divide_); + SetFastMethodNoSideEffect(context, + target, + "divide", + SlowDivide, + &fast_divide); } void RegisterExternalReferences(ExternalReferenceRegistry* registry) { registry->Register(SlowDivide); - registry->Register(FastDivide); - registry->Register(fast_divide_.GetTypeInfo()); + registry->Register(fast_divide); } } // namespace custom_namespace @@ -153,29 +488,11 @@ A typical function that communicates between JavaScript and C++ is as follows. node::custom_namespace::RegisterExternalReferences); ``` -* Update external references ([`node_external_reference.h`](../../src/node_external_reference.h)) - - Since our implementation used - `double(const int32_t a, const int32_t b, v8::FastApiCallbackOptions& options)` - signature, we need to add it to external references and in - `ALLOWED_EXTERNAL_REFERENCE_TYPES`. - - Example declaration: - - ```cpp - using CFunctionCallbackReturningDouble = double (*)(const int32_t a, - const int32_t b, - v8::FastApiCallbackOptions& options); - ``` - -### Test with Fast API path +* In the unit tests: -In debug mode (`./configure --debug` or `./configure --debug-node` flags), the -fast API calls can be tracked using the `TRACK_V8_FAST_API_CALL("key")` macro. -This can be used to count how many times fast paths are taken during tests. The -key is a global identifier and should be unique across the codebase. -Use `"binding_name.function_name"` or `"binding_name.function_name.suffix"` to -ensure uniqueness. + Since the Fast API callback uses `TRACK_V8_FAST_API_CALL`, we can ensure that + the fast paths are taken and test them by writing tests that force + V8 optimizations and check the counters. In the unit tests, since the fast API function uses `TRACK_V8_FAST_API_CALL`, we can ensure that the fast paths are taken and test them by writing tests that