Binding NCNN C API to MicroPython

[inspireMeNow]

This patch adds a complete MicroPython binding layer for the ncnn C API, exposing commonly used components (Option, Allocator, Mat, Net, Layer, Extractor, ParamDict, ModelBin, DataReader) together with pixel I/O, preprocessing, some utilities, and optional drawing support. Basic tests and documentation are included to demonstrate API usage and feature-configurable builds.

Core Design Principles

• Mirror ncnn object concepts instead of inventing Python abstractions.
• Prefer explicit, small functions over a large multi-purpose dispatcher.
• Fail fast: raise Python exceptions immediately on non-zero return codes or null object returns.
• Wrap C resources so they are released via Python’s gc finalizer, with optional xyz_destroy() methods for explicit cleanup.
• Keep build system dual-path (CMake + simple Make include) to ease integration into different MicroPython ports.

Changes Introduced

1. Build System Setup (CMake and Make)

The project allows compilation via CMake or Make. For this purpose, micropython.cmake and micropython.mk are provided, where the NCNN header files and libraries are explicitly specified.

2. Wrapping NCNN Functions with MicroPython

Implement ncnn_module.cpp, defining various mp_xyz() functions. Each function is wrapped for MicroPython using the MP_DEFINE_CONST_FUN_OBJ_ macro (with suffixes like _0, _1, _2 indicating the number of arguments). Functions that need to accept a variable number of arguments use MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN (with min/max bounds) or MP_DEFINE_CONST_FUN_OBJ_VAR (with only minimum). These function objects are then added to the module's global dictionary, e.g.,

static MP_DEFINE_CONST_FUN_OBJ_2(ncnn_net_get_input_index_obj, mp_ncnn_net_get_input_index);

3. Defining the Module Symbol Table

Define the module symbol table ncnn_module_globals_table and use the MP_DEFINE_CONST_DICT macro to create ncnn_module_globals. This dictionary contains all functions, constants, and classes exposed by the module.

4. Assigning the Globals Dictionary to the Module Object

Create the module object:

const mp_obj_module_t ncnn_user_cmodule = {
    .base = {&mp_type_module},
    .globals = (mp_obj_dict_t*)&ncnn_module_globals,
};

The .globals field of a MicroPython module object is a pointer to the dictionary that stores all the module’s attributes. MicroPython uses this dictionary to resolve module attributes such as functions and constants. Setting .globals is essential for exposing the module’s content to Python.

5. Registering the Module

The module is registered using the MP_REGISTER_MODULE macro:

MP_REGISTER_MODULE(MP_QSTR_ncnn, ncnn_user_cmodule);

This macro makes the ncnn module fully accessible from Python code. Internally, it registers the module and links its .globals to the previously defined dictionary, completing the binding.

Challenges & Resolutions

1. Handling Python Lists for Mean and Normalize Values

• Challenge:
  The ncnn_mat_substract_mean_normalize function requires arrays of floats for mean and normalization values, while in Python these are passed as lists (mp_obj_t). Correctly converting them is essential.

• Resolution:
  The binding extracts the Python list, converts items to floats, and allocates temporary arrays:

  mp_obj_t mean_list = args[1];
  size_t mean_len;
  mp_obj_t* mean_items;
  mp_obj_list_get(mean_list, &mean_len, &mean_items);
  
  float* mean_vals = (float*)malloc(mean_len * sizeof(float));
  for (size_t i = 0; i < mean_len; i++)
  {
      mean_vals[i] = (float)mp_obj_get_float(mean_items[i]);
  }

  After passing mean_vals and norm_vals to ncnn_mat_substract_mean_normalize, the arrays are freed:

  free(mean_vals);
  free(norm_vals);

2. Accessing Buffer Objects

• Challenge:
  NCNN functions operate on raw data buffers. MicroPython buffers (mp_obj_t) need to be accessed safely for reading or writing.

• Resolution:
  The binding uses mp_get_buffer_raise to access the buffer, specifying read or write access, and casts it to the proper type:

  mp_buffer_info_t bufinfo;
  mp_get_buffer_raise(args[0], &bufinfo, MP_BUFFER_READ);
  const unsigned char* pixels = (const unsigned char*)bufinfo.buf;

3. Understanding the Structure of Weight Arrays

• Problem:
  The binding function ncnn_modelbin_create_from_mat_array receives a weights and an integer n. Initially, it is unclear whether weights is a single matrix, a list of matrices, or something else, and what exactly n represents.

• Resolution:
  By reading the underlying implementation in src/c_api.cpp, it was confirmed that weights is a list of ncnn_mat_t objects (as integer handles) and n specifies the number of elements in this list:

ncnn_modelbin_t ncnn_modelbin_create_from_mat_array(const ncnn_mat_t* weights, int n)
{
    std::vector<Mat> matarray(n);
    for (int i = 0; i < n; i++)
    {
        matarray[i] = *(const Mat*)weights[i];
    }
    ncnn_modelbin_t mb = (ncnn_modelbin_t)malloc(sizeof(struct __ncnn_modelbin_t));
    mb->pthis = (void*)(new ModelBinFromMatArray_c_api(mb, n ? &matarray[0] : NULL));
    mb->load_1d = __ncnn_ModelBinFromMatArray_load_1d;
    mb->load_2d = __ncnn_ModelBinFromMatArray_load_2d;
    mb->load_3d = __ncnn_ModelBinFromMatArray_load_3d;
    return mb;
}

Thus, each element is retrieved with mp_obj_get_int and stored in a dynamically allocated C array before being passed to ncnn_modelbin_create_from_mat_array:

int n = mp_obj_get_int(n_obj);
size_t list_len;
mp_obj_t* list_items;
mp_obj_list_get(weights_obj, &list_len, &list_items);

if ((int)list_len != n)
{
    mp_raise_ValueError(MP_ERROR_TEXT("Array length mismatch with parameter n"));
}

ncnn_mat_t* weights = (ncnn_mat_t*)malloc(n * sizeof(ncnn_mat_t));
if (weights == NULL)
{
    mp_raise_msg(&mp_type_MemoryError, MP_ERROR_TEXT("Failed to allocate memory for weights array"));
}

for (int i = 0; i < n; i++)
{
    weights[i] = (ncnn_mat_t)mp_obj_get_int(list_items[i]);
}

Running Test Examples

Example test cases for the MicroPython NCNN binding can be found here.

$ micropython ./micropython/c_api/examples/test_all.py
NCNN MicroPython API Test
==================================================

=== VERSION API TESTS ===
NCNN Version: 1.0.20250818
[PASS] test_version_api: Completed successfully

=== ALLOCATOR API TESTS ===
Pool allocator created
Unlocked pool allocator created
Allocators destroyed
[PASS] test_allocator_api: Completed successfully

=== OPTION API TESTS ===
Option created
Threads set/get: 8
Use local pool allocator: 1
Blob allocator set
Workspace allocator set
Use Vulkan compute: 0
Option and allocators destroyed
[PASS] test_option_api: Completed successfully

=== MAT API TESTS ===
Empty mat created
1D mat created (100)
1D mat properties - dims: 1, w: 100, elemsize: 4, elempack: 1, cstep: 100
Mat filled and cloned
Mat reshaped to 1D (50), new width: 100
2D mat created (32x24)
2D mat height: 24
2D mat reshaped (16x48), new dims: 32x24
3D mat created (16x16x8)
3D mat depth: 1, channels: 8
3D mat reshaped, new dims: 3
Channel 0 data pointer: 591809408
4D mat created (8x8x4x2)
4D mat reshaped
External 1D mat created
1D elem mat created
2D elem mat created
3D elem mat created
4D elem mat created
[PASS] test_mat_api: Completed successfully

=== MAT PROCESS API TESTS ===
Mat substract_mean_normalize completed
Convert packing completed
Flatten completed
Copy make border completed
Copy cut border completed
Copy make border 3D completed
[PASS] test_mat_process_api: Completed successfully

=== PARAMDICT API TESTS ===
ParamDict created
Int param set/get: 42
Float param set/get: 3.141590118408203
Array param set/get: mat=591826512
Param types - 0: 2, 1: 3, 2: 4
ParamDict destroyed
[PASS] test_paramdict_api: Completed successfully

=== DATAREADER API TESTS ===
Basic DataReader created
Memory DataReader created
[PASS] test_datareader_api: Completed successfully

=== MODELBIN API TESTS ===
ModelBin from DataReader created
ModelBin from Mat array created
[PASS] test_modelbin_api: Completed successfully

=== LAYER API TESTS ===
Basic layer created
Layer created by type index: 591826512
Layer properties - typeindex: -1, one_blob_only: 0
Support - inplace: 0, vulkan: 0, packing: 0
Storage support - bf16: 0, fp16: 0
Layer properties set
Bottom count: 0, Top count: 0
Layers destroyed
[PASS] test_layer_api: Completed successfully

=== NET API TESTS ===
Network created
Network option retrieved: 591827080
Network option set
Input count: 0, Output count: 0
Network cleared
Network and option destroyed
[PASS] test_net_api: Completed successfully

=== PIXEL DRAWING API TESTS ===
draw_rectangle_c3 completed
draw_circle_c3 completed
draw_line_c3 completed
draw_text_c3 completed
[PASS] test_pixel_drawing_api: Completed successfully

=== PIXEL API TESTS ===
mat_from_pixels completed:
mat_to_pixels completed
mat_from_pixels_resize completed
mat_to_pixels_resize completed
mat_from_pixels_roi completed
mat_from_pixels_roi_resize completed
[PASS] test_pixel_api: Completed successfully

============================================================
TEST SUMMARY: 12 PASSED, 0 FAILED
Complete API Test Suite Finished

Practical Insights & Limitations

Working on this project required a amount of effort. On the surface, it mostly involves calling APIs, so the technical complexity may not seem very high. However, the implementation demands careful attention, especially when dealing with arrays, pointers, and error handling. Small mistakes in these areas can easily lead to runtime errors or memory issues, so meticulous coding and testing are essential.

However, a key limitation of the current binding is that C++ structures such as ncnn_mat_t and other related types are handled as raw integer handles. This approach is prone to memory leaks because Python's garbage collector cannot manage the lifecycle of these objects. Users must manually call cleanup methods (such as mat_destroy()) to properly release native resources.