Add docs about native extensions limitations

Author: msimacek

docs/user/Embedding-Permissions.md

@@ -56,7 +56,7 @@ The Java backend is the default when GraalPy is run via the `Context` API, that
 GraalPy can log information about known incompatibility of functions executed at runtime, which includes the OS interface-related functions.
 To turn on this logging, use the command-line option `--log.python.compatibility.level=FINE` (or other desired logging level).
 
-Known limitations of the of the Java backend are:
+Known limitations of the Java backend are:
 
 * Its state is disconnected from the actual OS state, which applies especially to:
   * *file descriptors*: Python-level file descriptors are not usable in native code.
@@ -73,4 +73,11 @@ Known limitations of the of the Java backend are:
 
 ## Python Native Extensions
 
-Python native extensions run by default as native binaries, with full access to the underlying system.
+Python native extensions run by default as native binaries, with full access to the underlying system. See [Embedding limitations](Native-Extensions.md#embedding-limitations)
+
+The context permissions needed to run native extensions are:
+```
+.allowIO(IOAccess.ALL)
+.allowCreateThread(true)
+.allowNativeAccess(true)
+```

docs/user/Native-Extensions.md

@@ -0,0 +1,31 @@
+---
+layout: docs-experimental
+toc_group: python
+link_title: Native Extensions Support
+permalink: /reference-manual/python/Native-Extensions/
+---
+
+# Native Extensions Support
+
+Python provides a native extensions API for writing Python extensions in C/C++. GraalPy's support for native extensions
+is currently considered experimental, although many packages like NumPy and PyTorch work well for many use cases.
+Native extensions built for CPython are not binary compatible with GraalPy, therefore it is not possible to use packages
+installed with CPython from GraalPy. Packages have to be installed with GraalPy. Likewise, prebuilt wheels for CPython
+from pypi.org cannot be used with GraalPy.
+The version of *pip* shipped with GraalPy applies additional patches to packages upon installation to make native
+extensions work, it is therefore crucial that you only use the *pip* preinstalled in GraalPy virtualenvs to install
+packages. Don't update *pip* or use alternative tools such as *uv*. GraalPy's *pip* is also preconfigured to use an
+extra repository from graalvm.org where we plan to publish prebuilt wheels for GraalPy for selected commonly used
+packages.
+
+## Embedding limitations
+
+Python native extensions run by default as native binaries, with full access to the underlying system.
+Native code is not sandboxed and can circumvent any protections Truffle or the JVM may provide, up to and including
+aborting the process.
+Native data structures are not subject to the Java GC and the combination of them with Java data structures may lead to
+memory leaks.
+Native libraries generally cannot be loaded multiple times into the same process, and they may contain global state that
+cannot be safely reset. Thus, it's not possible to create multiple GraalPy contexts that access native modules within
+the same JVM. This includes the case when you create a context, close it and then create another context. The second
+context will not be able to access native extensions.

graalpython/com.oracle.graal.python/src/com/oracle/graal/python/builtins/objects/cext/common/CExtContext.java

@@ -54,6 +54,7 @@
 import java.io.IOException;
 import java.nio.file.LinkOption;
 import java.util.Set;
+import java.util.logging.Level;
 
 import org.graalvm.collections.Pair;
 import org.graalvm.shadowed.com.ibm.icu.impl.Punycode;
@@ -304,15 +305,16 @@ private static String dlopenFlagsToString(int flags) {
     @TruffleBoundary
     public static Object loadCExtModule(Node location, PythonContext context, ModuleSpec spec, CheckFunctionResultNode checkFunctionResultNode)
                     throws IOException, ApiInitException, ImportException {
-
-        if (context.getOption(PythonOptions.WarnExperimentalFeatures) && !C_EXT_SUPPORTED_LIST.contains(spec.name.toJavaStringUncached())) {
-            getLogger().warning(() -> {
+        if (getLogger().isLoggable(Level.WARNING) && context.getOption(PythonOptions.WarnExperimentalFeatures)) {
+            boolean runViaLauncher = context.getOption(PythonOptions.RunViaLauncher);
+            if (!runViaLauncher || !C_EXT_SUPPORTED_LIST.contains(spec.name.toJavaStringUncached())) {
                 String message = "Loading C extension module %s from '%s'. Support for the Python C API is considered experimental.";
-                if (!context.getOption(PythonOptions.RunViaLauncher)) {
-                    message += " You can suppress this warning by setting the context option 'python.WarnExperimentalFeatures' to 'false'";
+                if (!runViaLauncher) {
+                    message += " See https://www.graalvm.org/latest/reference-manual/python/Native-Extensions/#embedding-limitations for the limitations. " +
+                                    "You can suppress this warning by setting the context option 'python.WarnExperimentalFeatures' to 'false'";
                 }
-                return message.formatted(spec.name, spec.path);
-            });
+                getLogger().warning(message.formatted(spec.name, spec.path));
+            }
         }
 
         // we always need to load the CPython C API