BLEClient heap memory exhausted after some disconnect/connect ricycle

[qrpfun]

Hi all,
I'm actually using official Arduino ESP32 library available from inside the Arduino IDE.
I need to connect via BT from an ESP32 to a peripheral made with an HC-08 connected to an arduino micro, all configured to sleep as much as possible to save energy (about 300uA in sleep), than read/write some info from there and close connection to get peripheral back in sleep mode, than reconnect only once needed and cycle this way over and over.

I'have started with BLE Client examples in the lib distribution but after some cycles everything crashes. After some debugging find out a problem with a heap memory exhaustion.

Just tried to do some research and some debugging but I would appreciate if someone would be so kind to get me some tips to find a way out from here.

this is the output from very first cycles where you can see rapid memory exhaustion:

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:10088
load:0x40080400,len:6380
entry 0x400806a4
BLE FreeHeap: 274408
Starting Arduino BLE Client application...
BLE Advertised Device found: Name: HC-08, Address: 88:4a:ea:8e:38:5d, manufacturer data: 48432d30380000000000000000000000000000000000, serviceUUID: 0000fff0-0000-1000-8000-00805f9b34fb, txPower: -6
BLE RSSI: -65
Forming a connection to 88:4a:ea:8e:38:5d
1-BLE FreeHeap: 181876
 - Created client
2-BLE FreeHeap: 181148
 - Connected to server
3-BLE FreeHeap: 177004
 - Found our service
4-BLE FreeHeap: 172504
 - Found our characteristic
5-BLE FreeHeap: 171076
We are now connected to the BLE Server.
Setting new characteristic value to "Time since boot: 2"
BLE FreeHeap: 171024
onDisconnect
BLE Advertised Device found: Name: HC-08, Address: 88:4a:ea:8e:38:5d, manufacturer data: 48432d30380000000000000000000000000000000000, serviceUUID: 0000fff0-0000-1000-8000-00805f9b34fb, txPower: -6
BLE RSSI: -66
Forming a connection to 88:4a:ea:8e:38:5d
1-BLE FreeHeap: 177236
 - Created client
2-BLE FreeHeap: 176508
 - Connected to server
3-BLE FreeHeap: 172360
 - Found our service
4-BLE FreeHeap: 167972
 - Found our characteristic
5-BLE FreeHeap: 166544
We are now connected to the BLE Server.
Setting new characteristic value to "Time since boot: 10"
BLE FreeHeap: 166492
....

and here the modified BLEClient.ino code:

/**
 * A BLE client example that is rich in capabilities.
 * There is a lot new capabilities implemented.
 * author unknown
 * updated by chegewara
 */

#include "BLEDevice.h"
//#include "BLEScan.h"

// The remote service we wish to connect to.
static BLEUUID serviceUUID("0000fff0-0000-1000-8000-00805f9b34fb");
static BLEUUID serviceIn_UUID("0000ffe0-0000-1000-8000-00805f9b34fb");
// The characteristic of the remote service we are interested in.
static BLEUUID    charUUID("0000ffe1-0000-1000-8000-00805f9b34fb");

static boolean doConnect = false;
static boolean connected = false;
static boolean doScan = false;
static BLERemoteCharacteristic* pRemoteCharacteristic;
static BLEAdvertisedDevice* myDevice;
static BLEClient*  pClient;
static BLERemoteService* pRemoteService;

static void notifyCallback(
  BLERemoteCharacteristic* pBLERemoteCharacteristic,
  uint8_t* pData,
  size_t length,
  bool isNotify) {
    Serial.print("Notify callback for characteristic ");
    Serial.print(pBLERemoteCharacteristic->getUUID().toString().c_str());
    Serial.print(" of data length ");
    Serial.println(length);
    Serial.print("data: ");
    Serial.println((char*)pData);
}

class MyClientCallback : public BLEClientCallbacks {
  void onConnect(BLEClient* pclient) {
  }

  void onDisconnect(BLEClient* pclient) {
    connected = false;
    Serial.println("onDisconnect");
//    free(pClient);
//    free(pRemoteService);
//    free(pRemoteCharacteristic);
//    delete(myDevice);
//    delete pClient; //crash
  }
};

bool connectToServer() {
    Serial.print("Forming a connection to ");
    Serial.println(myDevice->getAddress().toString().c_str());

    Serial.print("1-BLE FreeHeap: ");
    Serial.println(ESP.getFreeHeap());
    //BLEClient*  pClient  = BLEDevice::createClient();
    pClient  = BLEDevice::createClient();
    Serial.println(" - Created client");

    Serial.print("2-BLE FreeHeap: ");
    Serial.println(ESP.getFreeHeap());
    pClient->setClientCallbacks(new MyClientCallback());

    // Connect to the remove BLE Server.
    pClient->connect(myDevice);  // if you pass BLEAdvertisedDevice instead of address, it will be recognized type of peer device address (public or private)
    Serial.println(" - Connected to server");

    Serial.print("3-BLE FreeHeap: ");
    Serial.println(ESP.getFreeHeap());
    // Obtain a reference to the service we are after in the remote BLE server.
    //BLERemoteService* pRemoteService = pClient->getService(serviceIn_UUID);
    
    pRemoteService = pClient->getService(serviceIn_UUID);
    if (pRemoteService == nullptr) {
      Serial.print("Failed to find our service UUID: ");
      Serial.println(serviceUUID.toString().c_str());
      pClient->disconnect();
      return false;
    }
    Serial.println(" - Found our service");

    Serial.print("4-BLE FreeHeap: ");
    Serial.println(ESP.getFreeHeap());
    // Obtain a reference to the characteristic in the service of the remote BLE server.
    pRemoteCharacteristic = pRemoteService->getCharacteristic(charUUID);
    if (pRemoteCharacteristic == nullptr) {
      Serial.print("Failed to find our characteristic UUID: ");
      Serial.println(charUUID.toString().c_str());
      pClient->disconnect();
      return false;
    }
    Serial.println(" - Found our characteristic");

    Serial.print("5-BLE FreeHeap: ");
    Serial.println(ESP.getFreeHeap());
//    // BIG CRASH HERE - Read the value of the characteristic.
//    if(pRemoteCharacteristic->canRead()) {
//      std::string value = pRemoteCharacteristic->readValue();
//      Serial.print("The characteristic value was: ");
//      Serial.println(value.c_str());
//    }
//    if(pRemoteCharacteristic->canNotify())
//      pRemoteCharacteristic->registerForNotify(notifyCallback);

    connected = true;
    return true;
}
/**
 * Scan for BLE servers and find the first one that advertises the service we are looking for.
 */
class MyAdvertisedDeviceCallbacks: public BLEAdvertisedDeviceCallbacks {
 /**
   * Called for each advertising BLE server.
   */
  void onResult(BLEAdvertisedDevice advertisedDevice) {
    Serial.print("BLE Advertised Device found: ");
    Serial.println(advertisedDevice.toString().c_str());
    Serial.print("BLE RSSI: ");
    Serial.println(advertisedDevice.getRSSI() );

    // We have found a device, let us now see if it contains the service we are looking for.
    if (advertisedDevice.haveServiceUUID() && advertisedDevice.isAdvertisingService(serviceUUID)) {

      BLEDevice::getScan()->stop();
      myDevice = new BLEAdvertisedDevice(advertisedDevice);
      doConnect = true;
      doScan = true;

    } // Found our server
  } // onResult
}; // MyAdvertisedDeviceCallbacks


void setup() {
  Serial.begin(115200);
  Serial.print("BLE FreeHeap: ");
  Serial.println(ESP.getFreeHeap());
  Serial.println("Starting Arduino BLE Client application...");
  BLEDevice::init("");

  // Retrieve a Scanner and set the callback we want to use to be informed when we
  // have detected a new device.  Specify that we want active scanning and start the
  // scan to run for 5 seconds.
  BLEScan* pBLEScan = BLEDevice::getScan();
  pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks());
  pBLEScan->setInterval(1349);
  pBLEScan->setWindow(449);
  pBLEScan->setActiveScan(true);
  pBLEScan->start(5, false);
} // End of setup.


// This is the Arduino main loop function.
void loop() {

  // If the flag "doConnect" is true then we have scanned for and found the desired
  // BLE Server with which we wish to connect.  Now we connect to it.  Once we are 
  // connected we set the connected flag to be true.
  if (doConnect == true) {
    if (connectToServer()) {
      Serial.println("We are now connected to the BLE Server.");
    } else {
      Serial.println("We have failed to connect to the server; there is nothin more we will do.");
    }
    doConnect = false;
  }

  // If we are connected to a peer BLE Server, update the characteristic each time we are reached
  // with the current time since boot.
  if (connected) {
    String newValue = "Time since boot: " + String(millis()/1000);
    Serial.println("Setting new characteristic value to \"" + newValue + "\"");
    Serial.print("BLE FreeHeap: ");
    Serial.println(ESP.getFreeHeap());
    
    // Set the characteristic's value to be the array of bytes that is actually a string.
    //pRemoteCharacteristic->writeValue(newValue.c_str(), newValue.length());
    pRemoteCharacteristic->writeValue("LedOn", 5);
    delay(1000);
    pRemoteCharacteristic->writeValue("LedOn", 5);
    delay(2000);
    pRemoteCharacteristic->writeValue("LedOff", 6);
    delay(2000);
    pClient->disconnect();
    doConnect = false;
    
  }else if(doScan){
    BLEDevice::getScan()->start(0);  // this is just example to start scan after disconnect, most likely there is better way to do it in arduino
  }
  
  delay(1000); // Delay a second between loops.
} // End of loop

[chegewara]

Every time you call this pClient = BLEDevice::createClient(); you create new BLEClient, so you have to delete it somewhere in code after disconnect.
I have updated library to do it from onDisconnect() but in current library iyt will cause crash. What i can suggest now is something like that:

if(pClient != nullptr)
    delete(pClient); 
pClient  = BLEDevice::createClient();

[qrpfun]

chegewara many thanks for the ultra rapid answer!

Ok for the need to clear memory that way.
I have applied your suggestion and now a very strange behaviour occurs.
It runs a cycle ok, than when pointer is not clean the delete command crashes all as you can see the output here after. Mhh.. very critical

BLE FreeHeap: 274408
Starting Arduino BLE Client application...
BLE Advertised Device found: Name: , Address: 4a:61:ff:c1:be:1a, manufacturer data: 4c0010050a1c897627
BLE RSSI: -61
BLE Advertised Device found: Name: HC-08, Address: 88:4a:ea:8e:38:5d, manufacturer data: 48432d30380000000000000000000000000000000000, serviceUUID: 0000fff0-0000-1000-8000-00805f9b34fb, txPower: -6
BLE RSSI: -64
Forming a connection to 88:4a:ea:8e:38:5d
1-BLE FreeHeap: 181608
 - Created client
2-BLE FreeHeap: 180880
 - Connected to server
3-BLE FreeHeap: 176736
 - Found our service
4-BLE FreeHeap: 172244
 - Found our characteristic
5-BLE FreeHeap: 170816
We are now connected to the BLE Server.
Setting new characteristic value to "Time since boot: 1"
BLE FreeHeap: 170764
onDisconnect
BLE Advertised Device found: Name: HC-08, Address: 88:4a:ea:8e:38:5d, manufacturer data: 48432d30380000000000000000000000000000000000, serviceUUID: 0000fff0-0000-1000-8000-00805f9b34fb, txPower: -6
BLE RSSI: -64
Forming a connection to 88:4a:ea:8e:38:5d
1-BLE FreeHeap: 177248
CORRUPT HEAP: Bad head at 0x3ffe226c. Expected 0xabba1234 got 0x3ffe25b4
assertion "head != NULL" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/multi_heap_poisoning.c", line 214, function: multi_heap_free
abort() was called at PC 0x40149b97 on core 1

Backtrace: 0x4008d8d0:0x3ffc9f40 0x4008db01:0x3ffc9f60 0x40149b97:0x3ffc9f80 0x4009302d:0x3ffc9fb0 0x400846b2:0x3ffc9fd0 0x4008a3e5:0x3ffc9ff0 0x4000bec7:0x3ffca010 0x40090387:0x3ffca030 0x400d6923:0x3ffca050 0x400d46cf:0x3ffca070 0x400d4d99:0x3ffca090 0x400d5142:0x3ffca0b0 0x400d5161:0x3ffca0d0 0x400d34da:0x3ffca0f0 0x400d1732:0x3ffca120 0x400d1a5a:0x3ffca170 0x400d791d:0x3ffca210 0x40092545:0x3ffca230

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1

[chegewara]

Try to decode this backtrace:
Backtrace: 0x4008d8d0:0x3ffc9f40 0x4008db01:0x3ffc9f60 0x40149b97:0x3ffc9f80 0x4009302d:0x3ffc9fb0 0x400846b2:0x3ffc9fd0 0x4008a3e5:0x3ffc9ff0 0x4000bec7:0x3ffca010 0x40090387:0x3ffca030 0x400d6923:0x3ffca050 0x400d46cf:0x3ffca070 0x400d4d99:0x3ffca090 0x400d5142:0x3ffca0b0 0x400d5161:0x3ffca0d0 0x400d34da:0x3ffca0f0 0x400d1732:0x3ffca120 0x400d1a5a:0x3ffca170 0x400d791d:0x3ffca210 0x40092545:0x3ffca230

[chegewara]

Also you can try this code:
https://gist.github.com/chegewara/d3ed0e3365407f6d3abcffc486b2c462

[qrpfun]

I tried to substitute BLEClient.cpp with the one you suggested but no success
Same behaviour at the same place when deleting pClient

BLE FreeHeap: 171024
onDisconnect
BLE Advertised Device found: Name: HC-08, Address: 88:4a:ea:8e:38:5d, manufacturer data: 48432d30380000000000000000000000000000000000, serviceUUID: 0000fff0-0000-1000-8000-00805f9b34fb, txPower: -6
BLE RSSI: -66
Forming a connection to 88:4a:ea:8e:38:5d
1-BLE FreeHeap: 177432
CORRUPT HEAP: Bad head at 0x3ffe2148. Expected 0xabba1234 got 0x3ffe2490
assertion "head != NULL" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/multi_heap_poisoning.c", line 214, function: multi_heap_free
abort() was called at PC 0x40149c23 on core 1

Backtrace: 0x4008d8d0:0x3ffc9f20 0x4008db01:0x3ffc9f40 0x40149c23:0x3ffc9f60 0x4009302d:0x3ffc9f90 0x400846b2:0x3ffc9fb0 0x4008a3e5:0x3ffc9fd0 0x4000bec7:0x3ffc9ff0 0x40090387:0x3ffca010 0x400d699b:0x3ffca030 0x400d4747:0x3ffca050 0x400d4e11:0x3ffca070 0x400d51ba:0x3ffca090 0x400d51d9:0x3ffca0b0 0x400d33e6:0x3ffca0d0 0x400d3569:0x3ffca100 0x400d1772:0x3ffca120 0x400d1a9a:0x3ffca170 0x400d7995:0x3ffca210 0x40092545:0x3ffca230

Rebooting...
ets Jun  8 2016 00:22:57

[chegewara]

Hi,
i just uploaded code with few bug fixes. Could you try it?

[qrpfun]

do you suggest to git clone all the esp32-snippets/cpp_utils and use it instead of BLE lib inside arduino package or just last few specific file from this lib ?
Is my code right for the test or I have to change something ?
As soon I will be home I will try your code.
many thanks for your time

[chegewara]

git clone or download zip from this library and replace only those files which already exist in arduino library. There is no new file that needs to be added, just fixes.

[qrpfun]

Hi chegewara,
I have followed all you suggestions and it seems to work fine !!
Only pay attention to 2 wrong comments inside BLEDevice.cpp and BLEDevice.h where

static BLEServer*  m_pServer;
static BLEClient*   m_pClient;

I simply have removed comments and no problem to compile all.

it seems there is a remaining small memory leak but it may be caused by some minor problem in the code. I will clean the code and let it run all night and see is how long it keeps running and let you know.
By now many thanks for ultra kind support

Here is an output from the loop with memory check enabled:

**BLE FreeHeap: 169948**
onDisconnect
BLE Advertised Device found: Name: HC-08, Address: 88:4a:ea:8e:38:5d, manufacturer data: 48432d30380000000000000000000000000000000000, serviceUUID: 0000fff0-0000-1000-8000-00805f9b34fb, txPower: -6
BLE RSSI: -65
Forming a connection to 88:4a:ea:8e:38:5d
1-BLE FreeHeap: 176296
 - Created client
2-BLE FreeHeap: 179936
 - Connected to server
3-BLE FreeHeap: 175880
 - Found our service
4-BLE FreeHeap: 171412
 - Found our characteristic
5-BLE FreeHeap: 169984
We are now connected to the BLE Server.
Setting new characteristic value to "Time since boot: 424"
**BLE FreeHeap: 169932**
onDisconnect
BLE Advertised Device found: Name: , Address: 53:68:00:e5:48:ae, manufacturer data: 4c001005031804bc64
BLE RSSI: -64
BLE Advertised Device found: Name: HC-08, Address: 88:4a:ea:8e:38:5d, manufacturer data: 48432d30380000000000000000000000000000000000, serviceUUID: 0000fff0-0000-1000-8000-00805f9b34fb, txPower: -6
BLE RSSI: -67
Forming a connection to 88:4a:ea:8e:38:5d
1-BLE FreeHeap: 175996
 - Created client
2-BLE FreeHeap: 179644
 - Connected to server
3-BLE FreeHeap: 175584
 - Found our service
4-BLE FreeHeap: 171116
 - Found our characteristic
5-BLE FreeHeap: 169688
We are now connected to the BLE Server.
Setting new characteristic value to "Time since boot: 432"
**BLE FreeHeap: 169636**
onDisconnect

[chegewara]

Thanks for feedback.

[qrpfun]

OK and thanks again to you.
Here is an other issue that may be it should be better open in an other thread .. let me know.
The problem was present before updating library files and I just commented out those lines (you can see in the very first code submitted where was commented out)

here are related lines from BLEClient example:

    ....
    Serial.println(" - Found our characteristic");
    Serial.print("5-BLE FreeHeap: ");
    Serial.println(ESP.getFreeHeap());
    // BIG CRASH HERE - Read the value of the characteristic.
    if(pRemoteCharacteristic->canRead()) {
      std::string value = pRemoteCharacteristic->readValue(); // HERE HEAP CRASH
      Serial.print("The characteristic value was: ");
      Serial.println(value.c_str());
    }
    if(pRemoteCharacteristic->canNotify())
      pRemoteCharacteristic->registerForNotify(notifyCallback);
    connected = true;

And here the other related heap crash that happens just while reading characteristic:

4-BLE FreeHeap: 172000
 - Found our characteristic
5-BLE FreeHeap: 170576
CORRUPT HEAP: Bad head at 0x3ffe1da4. Expected 0xabba1234 got 0xad567800
assertion "head != NULL" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/multi_heap_poisoning.c", line 214, function: multi_heap_free
abort() was called at PC 0x4014a37b on core 0

Backtrace: 0x4008d8d0:0x3ffcf900 0x4008db01:0x3ffcf920 0x4014a37b:0x3ffcf940 0x4009302d:0x3ffcf970 0x400846b2:0x3ffcf990 0x4008a3e5:0x3ffcf9b0 0x4000bec7:0x3ffcf9d0 0x400d45b2:0x3ffcf9f0 0x400d5185:0x3ffcfa30 0x400d3abd:0x3ffcfa50 0x400d4435:0x3ffcfb00 0x400e3225:0x3ffcfb50 0x400df212:0x3ffcfba0 0x40092545:0x3ffcfbd0

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:10088
load:0x40080400,len:6380
entry 0x400806a4
BLE FreeHeap: 274384

[chegewara]

Yes, it would be good if you open new issue and try to encode backtrace too. Thanks.