Process Crash trying: .NET -> load ESM module -> loading .NET module

Beginne mit src/NodeApi/Runtime/NodeEmbedding.cs in der Nähe von ModuleInitializeCallbackAdapter und reproduziere den Absturz mithilfe der verlinkten minimalen C#-Konsolenanwendung. Vergleiche den Modules-Registrierungspfad mit dem ConfigureRuntime-Workaround und überprüfe, dass dynamische ESM-Imports mit process._linkedBinding nicht mehr den Prozess beenden oder einen Stackoverflow verursachen.

First, thanks for the amazing work!

I have a use-case where I want to:

• Load a dynamic ESM module from .NET
• Then "import" a .NET module into the ESM module.

Trying to do so using NodeEmbeddingRuntimeSettings.Modules and process._linkedBinding('<name>') crashes
the process immediately when calling process._linkedBinding.

Depending on where the lined binding is called the error is different.

• When calling process._linkedBinding in runtime.RunAsync the process terminates with error code 134 (0x86).

• When calling process._linkedBinding in MainScript I get a stack overflow.

More details and a possible fix/workaround down below.

TargetFramework: net9.0
Nuget: Microsoft.JavaScript.LibNode -version 20.1800.215
Nuget: Microsoft.JavaScript.NodeApi -version 0.9.17
OS: Microsoft Windows 11 Pro (10.0.26100 Build 26100)

Console App - Minimal reproduction

Demonstrating the Idea

The following is a fully made-up high-level example code to demonstrate what I want to do:

using var runtime = InitializeNodeEmbeddingThreadRuntime();

// Somehow register a .NET module:
runtime.RegisterModule("my-net-module", new MyNetModule()); // ⬅️ register a .NET module

// Somehow create, load and run an ES module that uses it:
var result = runtime.RunAsync(async () => {
    var exports = await runtime.LoadDynamicESModuleAsync("index.mjs", """
        import { add } from 'my-net-module';  // ⬅️ import the .NET module
        export const result = add(1, 2);
        """
    );
    return (int) exports.GetProperty("result");
});

Console.WriteLine($"Result: {result}");

public class MyNetModule {
    public int Add(int a, int b) => a + b;
}

What I came up with

• Use NodeEmbeddingModuleInfo to define a .NET module
• Use NodeEmbeddingRuntimeSettings.Modules to register the module.
• Use process._linkedBinding('<name>') to access the modules exports from js.

This crashes the process with the following error when loading the module:

  #  C:\Projects\Test\NodeJsTest\NodeJsTest\bin\Debug\net9.0\NodeJsTest.exe[113232]: void __cdecl node::embedding::EmbeddedRuntime::CloseV8Scope(unsigned __int64) at D:\a\_work\1\s\nodejs\src\node_embedding_api.cc:1345
  #  Assertion failed: (v8_scope_data_->nest_level()) == (nest_level)

----- Native stack trace -----

 1: 00007FFEF30E74F8 AES_cbc_encrypt+843048
 2: 00007FFEF31E8695 AES_cbc_encrypt+1896133
 3: 00007FFEF31D0733 AES_cbc_encrypt+1797987
 4: 00007FFEF31D0691 AES_cbc_encrypt+1797825
 5: 00007FFEEF6125D5

C:\Projects\Test\NodeJsTest\NodeJsTest\bin\Debug\net9.0\NodeJsTest.exe (process 113232) exited with code 134 (0x86).

Minimal reproduction:

// my custom .NET module:
var module = new NodeEmbeddingModuleInfo {
    Name = "example",
    NodeApiVersion = NodeEmbedding.NodeApiVersion,
    OnInitialize = (runtime, name, exports) => {
        exports["add"] = JSValue.CreateFunction("add", args => (int) args[0] + (int) args[1]);
        return exports;
    },
};

// Setup runtime and load the module:
var baseDir = AppContext.BaseDirectory;
var platform = new NodeEmbeddingPlatform(new());
using var runtime = platform.CreateThreadRuntime(baseDir, new NodeEmbeddingRuntimeSettings {
    MainScript = "globalThis.require = require('module').createRequire(process.execPath);",
    Modules = [module],
});

// Dynamically load an ESM module that uses the native module:
var result = await runtime.RunAsync(async () => {
    const string moduleName = "./index.mjs";
    File.WriteAllText(Path.Combine(baseDir, moduleName), """
        const { add } = process._linkedBinding("example");
        export const result = add(1, 2);
        """
    );

    var exports = await runtime.ImportAsync(moduleName, esModule: true);

    return (int) exports.GetProperty("result");
});

Console.WriteLine($"Result: {result}");

Next attempt

After reading the comment in the node_embedding_api.h I tried to load it in MainScript.

// Setup runtime and load the module:
var baseDir = AppContext.BaseDirectory;
var platform = new NodeEmbeddingPlatform(new());
using var runtime = platform.CreateThreadRuntime(baseDir, new NodeEmbeddingRuntimeSettings {
    MainScript = """
        globalThis.require = require('module').createRequire(process.execPath);
        globalThis.example = process._linkedBinding("example");
        """,
    Modules = [module],
});

// Dynamically load an ESM module that uses the native module:
var result = await runtime.RunAsync(async () => {
    const string moduleName = "./index.mjs";
    File.WriteAllText(Path.Combine(baseDir, moduleName), """
        export const result = globalThis.example.add(1, 2);
        """
    );
    ...

This results in a Stack overflow

Stack overflow.
   at System.Runtime.EH.DispatchEx(System.Runtime.StackFrameIterator ByRef, ExInfo ByRef)
   at System.Runtime.EH.RhThrowEx(System.Object, ExInfo ByRef)
   at Microsoft.JavaScript.NodeApi.JSValueScope.get_Current()
   at Microsoft.JavaScript.NodeApi.JSValue.GetCurrentRuntime(napi_env ByRef)
   at Microsoft.JavaScript.NodeApi.JSValue.CreateStringUtf16(System.String)
   at Microsoft.JavaScript.NodeApi.JSValue.op_Implicit(System.String)
   at Microsoft.JavaScript.NodeApi.JSError.CreateErrorValueForException(System.Exception, System.String ByRef)
   at Microsoft.JavaScript.NodeApi.JSError.ThrowError(System.Exception)
   at Microsoft.JavaScript.NodeApi.Runtime.NodeEmbedding.RuntimeLoadingCallbackAdapter(IntPtr, node_embedding_runtime, napi_env, napi_value, napi_value, napi_value)
   at Microsoft.JavaScript.NodeApi.Runtime.NodejsRuntime.EmbeddingCreateRuntime(node_embedding_platform, node_embedding_runtime_configure_callback, IntPtr, node_embedding_runtime ByRef)
   at Microsoft.JavaScript.NodeApi.Runtime.NodeEmbeddingRuntime.Create(Microsoft.JavaScript.NodeApi.Runtime.NodeEmbeddingPlatform, Microsoft.JavaScript.NodeApi.Runtime.NodeEmbeddingRuntimeSettings)
   at Microsoft.JavaScript.NodeApi.Runtime.NodeEmbeddingThreadRuntime+<>c__DisplayClass5_0.<.ctor>b__0()

C:\Projects\Test\NodeJsTest\NodeJsTest\bin\Debug\net9.0\NodeJsTest.exe (process 112360) exited with code -2147023895 (0x800703e9).

Fix and workaround

Looks like removing the new JSValueScope(JSValueScopeType.Root, env, JSRuntime) in ModuleInitializeCallbackAdapter solves both issues.

https://github.com/microsoft/node-api-dotnet/blob/bf474ba6fdb6fcda0e9b502ba5acf7f592eb053e/src/NodeApi/Runtime/NodeEmbedding.cs#L429-L439

Working code:

using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using Microsoft.JavaScript.NodeApi;
using Microsoft.JavaScript.NodeApi.Runtime;


// my custom .NET module:
var module = new NodeEmbeddingModuleInfo {
    Name = "example",
    NodeApiVersion = NodeEmbedding.NodeApiVersion,
    OnInitialize = (runtime, name, exports) => {
        exports["add"] = JSValue.CreateFunction("add", args => (int) args[0] + (int) args[1]);
        return exports; // side node: returning anything else then exports it self results in using that value as exports. tried `default`, `JSValue.Undefined`, and `JSValue.Null`.
    },
};

// Setup runtime and load the module:
var baseDir = AppContext.BaseDirectory;
var platform = new NodeEmbeddingPlatform(new());
using var runtime = platform.CreateThreadRuntime(baseDir, new NodeEmbeddingRuntimeSettings {
    MainScript = """
        globalThis.require = require('module').createRequire(process.execPath);
        globalThis.example = process._linkedBinding("example"); // ✔️ this works now!
        """,
    // Modules = [module],
    ConfigureRuntime = new ModuleRegistry([module]).ConfigureRuntime,
});

// Dynamically load an ESM module that uses the native module:
var result = await runtime.RunAsync(async () => {
    const string moduleName = "./index.mjs";
    File.WriteAllText(Path.Combine(baseDir, moduleName), """
    	const { add } = process._linkedBinding("example"); // ✔️ this also works!
        export const result = add(1, 2);
        """
    );

    var exports = await runtime.ImportAsync(moduleName, esModule: true);

    return (int) exports.GetProperty("result");
});

Console.WriteLine($"Result: {result}");


internal class ModuleRegistry(IEnumerable<NodeEmbeddingModuleInfo> modules) {
    public void ConfigureRuntime(NodejsRuntime.node_embedding_platform platform, NodejsRuntime.node_embedding_runtime_config config)
    {
        unsafe {
            foreach (var module in modules) {
                NodeEmbedding.Functor<NodejsRuntime.node_embedding_module_initialize_callback> functor = new() {
                    Data = (nint) GCHandle.Alloc(module.OnInitialize),
                    Callback = new NodejsRuntime.node_embedding_module_initialize_callback(&ModuleInitializeCallbackAdapter)
                };

                NodeEmbedding.JSRuntime.EmbeddingRuntimeConfigAddModule(
                        config,
                        module.Name.AsSpan(),
                        functor.Callback,
                        functor.Data,
                        functor.DataRelease,
                        module.NodeApiVersion ?? 0)
                    .ThrowIfFailed();
            }
        }
    }

    [UnmanagedCallersOnly(CallConvs = [typeof(CallConvCdecl)])]
    internal static unsafe JSRuntime.napi_value ModuleInitializeCallbackAdapter(
        nint cb_data,
        NodejsRuntime.node_embedding_runtime runtime,
        JSRuntime.napi_env env,
        nint module_name,
        JSRuntime.napi_value exports)
    {
    	// Removed the following line - everything else stays the same as used in `NodeEmbedding.cs`
        //❌ using var jsValueScope = new JSValueScope(JSValueScopeType.Root, env, NodeEmbedding.JSRuntime);
        try {
            var callback = (NodeEmbedding.InitializeModuleCallback) GCHandle.FromIntPtr(cb_data).Target!;
            NodeEmbeddingRuntime embeddingRuntime = NodeEmbeddingRuntime.FromHandle(runtime);
            return (JSRuntime.napi_value) callback(
                embeddingRuntime,
                Marshal.PtrToStringUTF8((nint) (byte*) module_name)!,
                new JSValue(exports));
        }
        catch (Exception ex) {
            JSError.ThrowError(ex);
            return JSRuntime.napi_value.Null;
        }
    }
}