BLE_HS_ENOMEM After Number of Indications Exceeds BT_NIMBLE_GATT_MAX_PROCS

[ryancog]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate
• Read the documentation to confirm the issue is not addressed there and your configuration is set correctly
• Tested with the latest version to ensure the issue hasn't been fixed

How often does this bug occurs?

always

Expected behavior

I'm running on ESP-IDF v5.4.2 (the latest version I see), and am experiencing an issue that I'm not entirely certain how to track down.

I have an esp32s3 configured as a BLE peripheral, connected to a phone application functioning as the central. I utilize a write/indicate model for accomplishing asynchronous esp32s3-side processing and data fetching, and so heavily utilize indications.

For one part of my project, I write a new value (a new name for the device), the device (esp32s3) is meant to update this value (which involves ble_svc_gap_device_name_set and an NVS store), and then ble_gatts_indicate after it has been updated.

The application subscribes to the indications as soon as it connects, and so upon receiving this new indication, it updates the value displayed in the UI.

Actual behavior (suspected bug)

Recently, upon a characteristic being written and the esp32s3 code completing the action and proceeding to indicate, the indication never reaches the phone application, and if I try to write trigger the action after this (accomplished via manual input in the phone app), after I try BT_NIMBLE_GATT_MAX_PROCS times, ble_gatts_indicate_custom() returns BLE_HS_ENOMEM. With NimBLE debug logging enabled, this final failed indication does not result in a "GATT procedure initiated" log.

Looking through the ble source and based on the behavior, my best guess is that it fails at ble_gattc_proc_alloc because, for whatever reason, those indications aren't being processed by NimBLE.

I'm not sure how best to troubleshoot this however, as I'm not intimately familiar with the NimBLE source.

Error logs or terminal output

Steps to reproduce the behavior

My application is not trivial at this point, and I'm currently working on trying to set up a somewhat workable MRE. For completion's sake here's the general outline:

1. Connect to esp32s3 running NimBLE with phone application (via flutter_blue_plus)
2. Subscribe to indications for name characteristic
3. Trigger name characteristic write with new value via application UI

• Indication is not received

4. Repeat until ENOMEM is returned.

I can share more information as requested, although I'm not sure what may be precisely useful.

Project release version

ESP-IDF v5.4.2

System architecture

Intel/AMD 64-bit (modern PC, older Mac)

Operating system

Linux

Operating system version

OpenSUSE Slowroll

Shell

ZSH

Additional context

This seems like it may be connected to #75, but there's not meaningful information there that is supplementary to my issue.

[rahult-github]

Hi @ryancog ,

Can you please try increasing the value of NIMBLE_GATT_MAX_PROCS from here during compilaiton and see if it helps your case.

[ryancog]

I have tried this. Increasing the value results in it simply requiring more attempts before ENOMEM is returned.

Similarly, the indication never seems to reach the phone app.

[ryancog]

I should note the rate of the indications is low, since I trigger them (more specifically the writes that result in indication) manually via GUI interaction. Once every few seconds is as fast as I can issue the write via my current UI.

[ryancog]

Quick update: it was a quirk of my app it appeared the indications were not received. That was not the case, and they are sent/received properly prior to ENOMEM.

Both in the app and doing some fiddling with nRF Connect I can see that the initial indication is received, as are subsequent indications up until ENOMEM is returned, and no further indications are sent.

[ryancog]

Additionally, I tested if it was an issue related to indicating within the GATT access callback by creating a new task for actually processing the received data (and ultimately indicating), however I observe the same behavior.

After BT_NIMBLE_GATT_MAX_PROCS indications, ENOMEM is returned.

[ryancog]

I have assembled a minimum reproducible example. After connecting with nRF Connect app and subscribing to indications, the full serial output is as follows:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce2820,len:0x51c
load:0x403c8700,len:0xf08
load:0x403cb700,len:0x31d4
entry 0x403c89fc
I (73) esp_psram: Found 2MB PSRAM device
I (73) esp_psram: Speed: 80MHz
I (74) cpu_start: Unicore app
I (74) cpu_start: Single core mode
I (90) cpu_start: Pro cpu start user code
I (90) cpu_start: cpu freq: 240000000 Hz
I (90) app_init: Application information:
I (93) app_init: Project name:     main
I (97) app_init: App version:      interact-core-34-g2b12d12-dirty
I (104) app_init: Compile time:     Jul  8 2025 13:55:23
I (110) app_init: ELF file SHA256:  87f1f59db...
I (115) app_init: ESP-IDF:          v5.4.2
I (120) efuse_init: Min chip rev:     v0.0
I (125) efuse_init: Max chip rev:     v0.99 
I (130) efuse_init: Chip rev:         v0.2
I (135) heap_init: Initializing. RAM available for dynamic allocation:
I (142) heap_init: At 3FCA3788 len 00045F88 (279 KiB): RAM
I (148) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (154) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM
I (160) heap_init: At 600FE04C len 00001F8C (7 KiB): RTCRAM
I (166) esp_psram: Adding pool of 2048K of PSRAM memory to heap allocator
I (174) spi_flash: detected chip: generic
I (178) spi_flash: flash io: qio
I (183) sleep_gpio: Configure to isolate all GPIO pins in sleep state
I (190) sleep_gpio: Enable automatic switching of GPIO sleep configuration
I (197) nvs_sec_provider: NVS Encryption - Registering HMAC-based scheme...
I (216) pm: Frequency switching config: CPU_MAX: 240, APB_MAX: 80, APB_MIN: 40, Light sleep: DISABLED
I (216) coexist: coex firmware version: 7b9a184
I (220) coexist: coexist rom version e7ae62f
I (225) main_task: Started on CPU0
I (225) main_task: Calling app_main()
I (235) nvs: NVS partition "nvs" is encrypted.
I (235) BLE_INIT: BT controller compile version [6cfabcd]
I (235) BLE_INIT: Bluetooth will use main XTAL as Bluetooth sleep clock.
I (245) BLE_INIT: Feature Config, ADV:1, BLE_50:1, DTM:1, SCAN:1, CCA:0, SMP:1, CONNECT:1
I (255) BLE_INIT: Bluetooth MAC: 98:3d:ae:f0:b1:ba
I (265) phy_init: phy_version 701,f4f1da3a,Mar  3 2025,15:50:10
I (285) BLE Test: Host Task Started.
I (305) NimBLE: GAP procedure initiated: stop advertising.

I (305) BLE Test: Synced
I (305) BLE Test: Device Address: BA:B1:F0:AE:3D:98
I (305) NimBLE: GAP procedure initiated: advertise; 
I (315) NimBLE: disc_mode=2
I (315) NimBLE:  adv_channel_map=0 own_addr_type=0 adv_filter_policy=0 adv_itvl_min=1216 adv_itvl_max=2056
I (325) NimBLE: 

I (325) BLE Test: Advertising started!
I (4285) NimBLE: GAP procedure initiated: 
I (4285) NimBLE: connection parameter update; conn_handle=1 itvl_min=12 itvl_max=12 latency=0 supervision_timeout=72 min_ce_len=0 max_ce_len=0
I (4295) NimBLE: 

BLE assert llc_con_upd.c 295, param 00000004 00000003
I (7785) NimBLE: GATT procedure initiated: indicate; 
I (7785) NimBLE: att_handle=3

I (7785) BLE Test: Indication completed with: 0
I (9285) NimBLE: GATT procedure initiated: indicate; 
I (9285) NimBLE: att_handle=3

I (9285) BLE Test: Indication completed with: 0
I (10785) NimBLE: GATT procedure initiated: indicate; 
I (10785) NimBLE: att_handle=3

I (10785) BLE Test: Indication completed with: 0
I (12285) NimBLE: GATT procedure initiated: indicate; 
I (12285) NimBLE: att_handle=3

I (12285) BLE Test: Indication completed with: 0
I (13785) BLE Test: Indication completed with: 6
I (15285) BLE Test: Indication completed with: 6
I (16785) BLE Test: Indication completed with: 6
I (18285) BLE Test: Indication completed with: 6

As you can see: in this case BT_NIMBLE_GATT_MAX_PROCS was set to 4, so after 4 indications, it fails with 6 (BLE_HS_ENOMEM).

Below is the complete MRE file. There's many shortcuts taken for the sake of being relatively simple and small, but the behavior even in this state is identical to what I'm seeing in my full app.

#include <array>

#include <esp_event.h>
#include <host/ble_hs.h>
#include <host/util/util.h>
#include <nimble/nimble_port.h>
#include <nvs_flash.h>
#include <services/gap/ble_svc_gap.h>

// This is extern somewhere?
// NOLINTNEXTLINE(readability-identifier-naming, modernize-redundant-void-arg) 
extern "C" void ble_store_config_init(void);

// "Unknown"
constexpr auto APPEARANCE_VAL{0};

constexpr const char *TAG{"BLE Test"};

std::array<StackType_t, 8192> hostStack;
StaticTask_t hostTaskData;

void syncCB();
void resetCB(int reason);
void hostTask(void *param);
void startAdvertising();

uint16_t connHandle;
uint8_t ownAddressType;
std::array<uint8_t, 6> addressValue;

constexpr ble_uuid128_t UUID = BLE_UUID128_INIT(
        0x00, 0x11, 0x00, 0x11, 0x00, 0x11, 0x00, 0x11,
        0x00, 0x11, 0x00, 0x11, 0x00, 0x11, 0x00, 0x11);

uint16_t charHandle;
bool charShouldIndicate;

void formatAddr(std::array<char, 18>& addrStr, const uint8_t addr[]) {
    (void)snprintf(addrStr.data(), addrStr.size(), "%02X:%02X:%02X:%02X:%02X:%02X", addr[0], addr[1],
            addr[2], addr[3], addr[4], addr[5]);
    addrStr.back() = '\0';
}

void printConnDesc(const ble_gap_conn_desc& desc) {
    std::array<char, 18> addrStr;

    ESP_LOGV(TAG, "Connection Handle: %u", desc.conn_handle);

    formatAddr(addrStr, desc.our_id_addr.val);
    ESP_LOGV(TAG, "Device ID Addr: Type: %u Value: %s", static_cast<uint32_t>(desc.our_id_addr.type), addrStr.data());

    formatAddr(addrStr, desc.peer_id_addr.val);
    ESP_LOGV(TAG, "Peer ID Addr: Type: %u Value: %s", static_cast<uint32_t>(desc.peer_id_addr.type), addrStr.data());

    ESP_LOGV(TAG,
            "Conn Itvl: %u Conn Latency: %u Supervision Timeout: %u Encrypted: %u Authenticated: %u Bonded: %u", desc.conn_itvl, desc.conn_latency, 
            desc.supervision_timeout, desc.sec_state.encrypted, desc.sec_state.authenticated, desc.sec_state.bonded);
}


int32_t handleConnection(const decltype(ble_gap_event::connect)& conn) {
    ESP_LOGV(TAG, "Connection %s%d", conn.status == 0 ? "established: " : "failed: ", conn.status);

    if (conn.status != 0) {
        // Connection failed
        startAdvertising();
        return 0;
    }

    int32_t res{};

    // Check connection handle
    ble_gap_conn_desc desc;
    res = ble_gap_conn_find(conn.conn_handle, &desc);
    if (res != 0) {
        ESP_LOGE(TAG, "Failed to find connection via handle: %d", res);
        return res;
    }

    // Attempt to update to client desired parameters
    // See: https://developer.apple.com/library/archive/qa/qa1931/_index.html
    struct ble_gap_upd_params params = {
        .itvl_min = 12,
        .itvl_max = 12,
        .latency = desc.conn_latency,
        .supervision_timeout = desc.supervision_timeout,
        .min_ce_len = 0,
        .max_ce_len = 0,
    };
    res = ble_gap_update_params(conn.conn_handle, &params);
    if (res != 0) {
        ESP_LOGE(TAG, "Failed to update connection parameters: %d", res);
        return res;
    }

    connHandle = conn.conn_handle;
    ble_gap_conn_find(connHandle, &desc);
    printConnDesc(desc);

    return 0;
}

int32_t handleDisconnection(const decltype(ble_gap_event::disconnect)& disconn) {
    ESP_LOGI(TAG, "Disconnected from board: %d", disconn.reason);

    connHandle = BLE_HS_CONN_HANDLE_NONE;
    startAdvertising();
    return 0;
}

ssize_t advertisementCB(ble_gap_event *event, void *) {
    switch (event->type) {
        case BLE_GAP_EVENT_CONNECT:
            return handleConnection(event->connect);
        case BLE_GAP_EVENT_DISCONNECT:
            return handleDisconnection(event->disconnect);
        case BLE_GAP_EVENT_SUBSCRIBE:
            charShouldIndicate = event->subscribe.cur_indicate;
            return 0;
        case BLE_GAP_EVENT_CONN_UPDATE:
        case BLE_GAP_EVENT_PHY_UPDATE_COMPLETE: 
        case BLE_GAP_EVENT_ENC_CHANGE:
        case BLE_GAP_EVENT_NOTIFY_RX:
        case BLE_GAP_EVENT_NOTIFY_TX:
        case BLE_GAP_EVENT_LINK_ESTAB:
        case BLE_GAP_EVENT_CONN_UPDATE_REQ:
        case BLE_GAP_EVENT_MTU: 
        case BLE_GAP_EVENT_DATA_LEN_CHG:
            // Ignored events
            return 0;
        default:
            ESP_LOGW(TAG, "Unhandled case in advertisement CB: %u", static_cast<uint32_t>(event->type));
    }

    return 0;
}

void startAdvertising() {
    if (not ble_hs_is_enabled()) return;
    if (ble_gap_adv_active()) return;
    ESP_LOGV(TAG, "Starting advertising...");

    /* Local variables */
    int32_t res{0};
    ble_hs_adv_fields advFields;
    ble_hs_adv_fields rspFields;
    ble_gap_adv_params advParams;

    bzero(&advFields, sizeof advFields);
    bzero(&rspFields, sizeof rspFields);
    bzero(&advParams, sizeof advParams);

    advFields.flags = BLE_HS_ADV_F_DISC_GEN | BLE_HS_ADV_F_BREDR_UNSUP;

    const char *name{ble_svc_gap_device_name()};
    advFields.name = reinterpret_cast<const uint8_t *>(name);
    advFields.name_len = strnlen(name, 8);
    advFields.name_is_complete = 0;

    advFields.uuids128 = &UUID;
    advFields.num_uuids128 = 1;
    advFields.uuids128_is_complete = true;

    res = ble_gap_adv_set_fields(&advFields);
    assert(res == 0);

    rspFields.name = reinterpret_cast<const uint8_t *>(name);
    rspFields.name_len = strlen(name);
    rspFields.name_is_complete = true;

    res = ble_gap_adv_rsp_set_fields(&rspFields);
    assert(res == 0 or res == BLE_HS_EMSGSIZE);

    advParams.conn_mode = BLE_GAP_CONN_MODE_UND;
    advParams.disc_mode = BLE_GAP_DISC_MODE_GEN;

    advParams.itvl_min = BLE_GAP_ADV_ITVL_MS(760);
    advParams.itvl_max = BLE_GAP_ADV_ITVL_MS(1285);

    /* Start advertising */
    res = ble_gap_adv_start(ownAddressType, nullptr, BLE_HS_FOREVER, &advParams, advertisementCB, nullptr);
    assert(res == 0);

    ESP_LOGI(TAG, "Advertising started!");
}

void syncCB() {
    ESP_LOGI(TAG, "Synced");
    int32_t res{};

    res = ble_hs_util_ensure_addr(0);
    assert(res == 0);

    res = ble_hs_id_infer_auto(0, &ownAddressType);
    assert(res == 0);

    res = ble_hs_id_copy_addr(ownAddressType, addressValue.data(), nullptr);
    assert(res == 0);

    std::array<char, 18> addrStr;
    formatAddr(addrStr, addressValue.data());
    ESP_LOGI(TAG, "Device Address: %s", addrStr.data());

    startAdvertising();
}

void resetCB(int reason) { ESP_LOGI(TAG, "Reset: %d", reason); }

void hostTask(void *) {
    ESP_LOGI(TAG, "Host Task Started.");
    // Returns only after nimble_port_stop()
    nimble_port_run(); 
    vTaskDelete(nullptr);
}

ssize_t accessCB(uint16_t /*conn_handle*/, uint16_t /*attr_handle*/, ble_gatt_access_ctxt *ctxt, void * /*arg*/) {
    if (ctxt->op == BLE_GATT_ACCESS_OP_READ_CHR) {
        constexpr std::array<uint8_t, 4> DATA{0x00, 0x01, 0x10, 0x11};
        os_mbuf_append(ctxt->om, DATA.data(), DATA.size());
    } else {
        ESP_LOGE(TAG, "BLE wanted unsupported access type: %u", ctxt->op);
    }

    return 0;
}

constexpr ble_gatt_svc_def NULL_SERVICE{
    .type = BLE_GATT_SVC_TYPE_END,
    .uuid = nullptr,
    .includes = nullptr,
    .characteristics = nullptr,
};
constexpr ble_gatt_chr_def NULL_CHARACTERISTIC{
    .uuid = nullptr,
    .access_cb = nullptr,
    .arg = nullptr,
    .descriptors = nullptr,
    .flags = 0,
    .min_key_size = 0,
    .val_handle = nullptr,
    .cpfd = nullptr,
};

// C-style API, c-style arrays :/
const ble_gatt_svc_def GATT_SERVICES[]{
    {
        .type = BLE_GATT_SVC_TYPE_PRIMARY,
        .uuid = &UUID.u,
        .includes = (const ble_gatt_svc_def *[]){nullptr},
        .characteristics = (const ble_gatt_chr_def[]) {
            ble_gatt_chr_def{
                .uuid = &UUID.u,
                .access_cb = accessCB,
                .arg = nullptr,
                .descriptors = nullptr,
                .flags = BLE_GATT_CHR_F_READ | BLE_GATT_CHR_F_INDICATE,
                .min_key_size = 0,
                .val_handle = &charHandle,
                .cpfd = nullptr
            },
            NULL_CHARACTERISTIC
        },
    },
    NULL_SERVICE
};

// NOLINTNEXTLINE(readability-identifier-naming)
extern "C" void app_main() {
    int32_t res{};

    res = esp_event_loop_create_default();
    assert(res == ESP_OK);

    res = nvs_flash_init();
    if (
            res == ESP_ERR_NVS_NO_FREE_PAGES or
            res == ESP_ERR_NVS_NEW_VERSION_FOUND
       ) {
        ESP_ERROR_CHECK(nvs_flash_erase());
        res = nvs_flash_init();
    }
    assert(res == ESP_OK);

    res = nimble_port_init();
    assert(res == ESP_OK);

    ble_hs_cfg.sm_io_cap = BLE_SM_IO_CAP_NO_IO;
    ble_hs_cfg.sm_mitm = 1;
    ble_hs_cfg.sm_sc = 1;
    ble_hs_cfg.sm_bonding = 0;

    ble_hs_cfg.reset_cb = resetCB;
    ble_hs_cfg.sync_cb = syncCB;
    ble_hs_cfg.store_status_cb = ble_store_util_status_rr;
    ble_store_config_init();

    // GATT Initialization
    ble_gattc_init();

    res = ble_gatts_count_cfg(GATT_SERVICES);
    assert(res == 0);

    res = ble_gatts_add_svcs(GATT_SERVICES);
    assert(res == 0);

    // GAP Initialization
    ble_svc_gap_init();

    res = ble_svc_gap_device_name_set("Test Device");
    assert(res == 0);

    // See https://developer.nordicsemi.com/nRF5_SDK/nRF51_SDK_v4.x.x/doc/html/group___b_l_e___a_p_p_e_a_r_a_n_c_e_s.html
    res = ble_svc_gap_device_appearance_set(APPEARANCE_VAL);
    assert(res == 0);

    xTaskCreateStaticPinnedToCore(hostTask, "NimBLE Host", hostStack.size(), nullptr, 5, hostStack.data(), &hostTaskData, 0);

    while (true) {
        if (charShouldIndicate) {
            res = ble_gatts_indicate(connHandle, charHandle);
            ESP_LOGI(TAG, "Indication completed with: %d", res);
        }
        vTaskDelay(150);
    }
}

[ryancog]

@rahult-github please let me know if you are able to reproduce this issue. I can additionally provide my sdkconfig settings if they seem to be relevant.

[rahult-github]

Hi @ryancog ,

can you kindly confirm the below print is always seen at your end whenever issue happens, or is it just this particular log?

BLE assert llc_con_upd.c 295, param 00000004 00000003

[ryancog]

@rahult-github That log consistently shows upon first connection to my iOS device. I had not noticed it before, but it appears in my real esp32s3 code output as well.

However, it does not appear when connecting to the Android device, and the same indication issue presents itself regardless of which device is connected, so it seems unrelated.

[ryancog]

Here I have my sdkconfig I used for the MRE

sdkconfig.txt

[ryancog]

Hi @rahult-github,

Could you let me know if you're able to reproduce this issue, or if there's more information you need from me? Are there sections of the code you would recommend I investigate to try and find the source of this issue? TIA

[rahult-github]

The code changes you provided are giving compilation issues .. I guess they can't be used as is. However , i think i understand what issue you are facing. Will check on it.

[ryancog]

I appreciate you checking.

For the code, perhaps I overlooked something, that code should be identical (direct copy/past) to what I uploaded. I'll definitely take another look and see if I missed something.

[ryancog]

Hello @rahult-github, any luck on this? Any leads I can chase down on my end?

[buganini]

I have the same issue with
v5.5
v5.4.2

It was fine with
v5.4.1
v5.4