Using omblepy implementation for supporting OMRON devices in Gadgetbridge

[corrados]

I have a HEM-7600T but I am not able to readout data with omblepy. I tried it under Ubuntu, ESP32 and Windows 10. Today, I once could readout data with UBPM under Windows but cannot reproduce it anymore :-/. So, an alternative solution would be great.

As you already tried to get rid of OS restrictions with regard to bluetooth with your ESP32 implementation, another solution would be to integrate your work in Gadgetbridge which runs on Android. There is already an open issue at Gadgetbridge for it: https://codeberg.org/Freeyourgadget/Gadgetbridge/issues/3656

Would that be something you would consider supporting?

[userx14]

Hi corrados,

Gadgetbridge looks like nice project, but unfortunatelly I'm lacking the java skills and time to work on that.
If there are any questions on the protocol or pairing procedure I'm happy to help anyone attempting to implement this.

For win10 you really need to be fast with the pairing dialog confirmation.
Unpairing the omron from the list of previously paired devices, also seems to be required.

Do you get any kind of error from omblepy while trying to pair / connect?

Did you already test with the esp32 bridge?

Best,
Benjamin

[corrados]

Hi Benjamin,

sorry for my late response but I was not at home for several days and could not access my Omron device.

Gadgetbridge looks like nice project, but unfortunatelly I'm lacking the java skills and time to work on that.

Same with me. I code in C++ but I do not have any java skills too, unfortunately...

For win10 you really need to be fast with the pairing dialog confirmation.

That did the trick! :-) If I very quickly select the device from the list when trying to pair, it works.

Now I am using the -m flag for reading data and it works perfectly.

Thank you for your support.

[corrados]

Did you already test with the esp32 bridge?

Yes I did but that did not work. Do you want me to try it again and give you more info on the error message?

[userx14]

Nice that you get the windows version working. I also looked into the short timeout issue, but was unable to figure out where this goes wrong.

Did you already test with the esp32 bridge?

Yes I did but that did not work. Do you want me to try it again and give you more info on the error message?

If you find the time for futher testing with esp32, it would be useful to know in which stage the error occurs.
You can test the pairing phase with just the arduino IDE and it's serial console, using the commands listed here.

Thanks a lot.

[corrados]

It's a bit hard for me to find out what goes wrong with the ESP32 connection. When I tried first, I did not see the device in the list:

python3 omblepy_bridge.py -b /dev/ttyUSB0 -p -d HEM-7600T --loggerDebug
2024-10-01 18:31:35,835 - omblepy - INFO - Attempt to import module for device hem-7600t
To improve your chance of a successful connection please do the following:
-remove previous device pairings in your OS's bluetooth dialog
-enable bluetooth on you omron device and use the specified mode (pairing or normal)
-do not accept any pairing dialog until you selected your device in the following list

+----+-----+------+------+
| ID | MAC | NAME | RSSI |
+----+-----+------+------+
Enter ID or just press Enter to rescan.

After rebooting the ESP32, it then found it:

python3 omblepy_bridge.py -b /dev/ttyUSB0 -p -d HEM-7600T --loggerDebug
2024-10-01 18:33:03,759 - omblepy - INFO - Attempt to import module for device hem-7600t
To improve your chance of a successful connection please do the following:
-remove previous device pairings in your OS's bluetooth dialog
-enable bluetooth on you omron device and use the specified mode (pairing or normal)
-do not accept any pairing dialog until you selected your device in the following list

+----+-------------------+-------------------------------+------+
| ID | MAC | NAME | RSSI |
+----+-------------------+-------------------------------+------+
| 0 | 00:5f:bf:xx:xx:xx | BLEsmart_0000021F005XXXXXXXXX | -37 |
| 1 | 72:fd:85:xx:xx:xx | | -53 |
| 2 | 54:3f:bd:xx:xx:xx | | -94 |
+----+-------------------+-------------------------------+------+
Enter ID or just press Enter to rescan.
0
2024-10-01 18:33:07,950 - omblepy - INFO - Attempt connecting to 00:5f:bf:xx:xx:xx.
2024-10-01 18:33:17,960 - omblepy - INFO - unpair and disconnect
Traceback (most recent call last):
File "/home/corrados/TMP/omblepy/omblepy_bridge.py", line 280, in
main()
File "/home/corrados/TMP/omblepy/omblepy_bridge.py", line 260, in main
raise ValueError(f"pairing failed {response}")
ValueError: pairing failed b''

Using my known MAC address, it says "communication started" but there is no file with values:

python3 omblepy_bridge.py -b /dev/ttyUSB0 -d HEM-7600T --mac 00:5F:BF:x:x:x --loggerDebug
2024-10-01 18:33:30,625 - omblepy - INFO - Attempt to import module for device hem-7600t
2024-10-01 18:33:30,631 - omblepy - INFO - Attempt connecting to 00:5F:BF:x:x:x.
2024-10-01 18:33:30,631 - omblepy - INFO - communication started
2024-10-01 18:33:40,636 - omblepy - INFO - unpair and disconnect
Traceback (most recent call last):
File "/home/corrados/TMP/omblepy/omblepy_bridge.py", line 280, in
main()
File "/home/corrados/TMP/omblepy/omblepy_bridge.py", line 270, in main
raise ValueError(f"connection failed {response}")
ValueError: connection failed b''

Does this help?

[userx14]

Not sure why it needs a restart of the esp32, but I can see that even the pairing process is not working.
Could you reupload the arduino esp32 code, with 'Tools->Core Debug Level->Info' set?
This will break the communication with the python script, as it generates additional messages which can not be parsed by omblepy.

Enable the pairing mode of the omron device and send the esp32 the letter "S" in the Arduino IDE serial console, to check that it is visible. Then try to pair by sending 'p 00:5F:BF:xx:xx:xx'.
This should log some of the communication to the serial console, which could help to pin down the problem.

Thanks.

[corrados]

Here is what I get:

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:4604
ho 0 tail 12 room 4
load:0x40078000,len:15488
load:0x40080400,len:4
load:0x40080404,len:3180
entry 0x400805b8
[ 16][I][esp32bridge.ino:115] setup(): [ESP32OmBlePy] ESP32 bridge online
[ 689][I][esp32bridge.ino:121] setup(): [ESP32OmBlePy] try bonding
[ 16753][I][esp32bridge.ino:257] loop(): [ESP32OmBlePy] Trying to connect to 00:5f:bf:x:x:x
[ 16762][I][BLEDevice.cpp:598] addPeerDevice(): add conn_id: 0, GATT role: client
[ 46775][E][BLEClient.cpp:249] gattClientEventHandler(): Failed to connect, status=Unknown ESP_ERR error
[ 46785][I][BLEDevice.cpp:609] removePeerDevice(): remove: 0, GATT role client
[ 46792][I][esp32bridge.ino:149] writeNewUnlockKey(): [ESP32OmBlePy] get service

[userx14]

So the issue seems to be before writing to any bluetooth characteristic, in the bluetooth bonding phase with the esp32.
This could be caused from the 7600t expecting different security settings for the bluetooth connection.
The part of the arduino code that setups this is here:

omblepy/esp32bridge/esp32bridge.ino

Lines 117 to 126 in 406f706

	BLEDevice::setEncryptionLevel(ESP_BLE_SEC_ENCRYPT);
	BLEDevice::setSecurityCallbacks(new OmronBleSecCallbacks());
	//setup encryption for successful pairing
	BLESecurity *securityP = new BLESecurity();
	ESP_LOGI(LOG_TAG, "try bonding");
	securityP->setCapability(ESP_IO_CAP_NONE);
	securityP->setKeySize(16);
	securityP->setAuthenticationMode(ESP_LE_AUTH_BOND); //also available ESP_LE_AUTH_NO_BOND, ESP_LE_AUTH_REQ_SC_MITM, ESP_LE_AUTH_BOND, ESP_LE_AUTH_NO_BOND
	securityP->setInitEncryptionKey(ESP_BLE_ENC_KEY_MASK | ESP_BLE_ID_KEY_MASK);
	securityP->setRespEncryptionKey(ESP_BLE_ENC_KEY_MASK | ESP_BLE_ID_KEY_MASK);

Here is some information on these settings:
https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/bluedroid/ble/gatt_security_server/tutorial/Gatt_Security_Server_Example_Walkthrough.md

I will try to record the output for pairing with my 7322t, to have a better comparison.

[userx14]

So this is how it does look like for my hem7322t for comparison:

Arduino Log with Info

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 188777542, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:4604
ho 0 tail 12 room 4
load:0x40078000,len:15488
load:0x40080400,len:4
load:0x40080404,len:3180
entry 0x400805b8
[    38][I][esp32bridge.ino:115] setup(): [ESP32OmBlePy] ESP32 bridge online
[   754][I][esp32bridge.ino:121] setup(): [ESP32OmBlePy] try bonding
[  7799][I][esp32bridge.ino:257] loop(): [ESP32OmBlePy] Trying to connect to d3:07:19:XX:XX:XX
[  7807][I][BLEDevice.cpp:598] addPeerDevice(): add conn_id: 0, GATT role: client
[  8447][I][esp32bridge.ino:149] writeNewUnlockKey(): [ESP32OmBlePy] get service
[  8525][I][BLEDevice.cpp:231] gapEventHandler(): ESP_GAP_BLE_AUTH_CMPL_EVT
[  8532][I][esp32bridge.ino:102] onAuthenticationComplete(): [ESP32OmBlePy] Auth success, remote BD_ADDR:
[  8541][I][esp32bridge.ino:104] onAuthenticationComplete(): [ESP32OmBlePy] address type = 1
[  8549][I][esp32bridge.ino:105] onAuthenticationComplete(): [ESP32OmBlePy] auth mode = 1
[ 13602][I][esp32bridge.ino:155] writeNewUnlockKey(): [ESP32OmBlePy] get characteristic
[ 13611][E][BLERemoteCharacteristic.cpp:287] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
[ 13621][E][BLERemoteCharacteristic.cpp:287] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
[ 13631][E][BLERemoteCharacteristic.cpp:287] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
[ 13641][E][BLERemoteCharacteristic.cpp:287] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
[ 13798][I][esp32bridge.ino:165] writeNewUnlockKey(): [ESP32OmBlePy] write unlock code
[ 13897][I][esp32bridge.ino:175] writeNewUnlockKey(): [ESP32OmBlePy] write default key
p OK
...

Arduino Log with Verbose

...
[ 10562][V][BLEDevice.cpp:62] createClient(): >> createClient
[ 10568][V][BLEDevice.cpp:68] createClient(): << createClient
[ 10574][I][esp32bridge.ino:257] loop(): [ESP32OmBlePy] Trying to connect to d3:07:19:xx:xx:xx
[ 10582][V][BLEClient.cpp:110] connect(): >> connect(d3:07:19:xx:xx:xx)
[ 10589][I][BLEDevice.cpp:598] addPeerDevice(): add conn_id: 0, GATT role: client
[ 10596][V][FreeRTOS.cpp:179] take(): Semaphore taking: name: RegEvt (0x3ffdcdcc), owner: <N/A> for connect
[ 10606][V][FreeRTOS.cpp:188] take(): Semaphore taken:  name: RegEvt (0x3ffdcdcc), owner: connect
[ 10614][V][FreeRTOS.cpp:59] wait(): >> wait: Semaphore waiting: name: RegEvt (0x3ffdcdcc), owner: connect for connect
[ 10615][D][BLEDevice.cpp:138] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... ESP_GATTC_REG_EVT
[ 10635][V][BLEUtils.cpp:1202] dumpGattClientEvent(): GATT Event: ESP_GATTC_REG_EVT
[ 10643][V][BLEUtils.cpp:1357] dumpGattClientEvent(): [status: ESP_GATT_OK, app_id: 0x0]
[ 10651][D][BLEClient.cpp:185] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... ESP_GATTC_REG_EVT
[ 10661][V][FreeRTOS.cpp:136] give(): Semaphore giving: name: RegEvt (0x3ffdcdcc), owner: connect
[ 10670][V][FreeRTOS.cpp:73] wait(): << wait: Semaphore released: name: RegEvt (0x3ffdcdcc), owner: <N/A>
[ 10679][V][FreeRTOS.cpp:179] take(): Semaphore taking: name: OpenEvt (0x3ffdd298), owner: <N/A> for connect
[ 10689][V][FreeRTOS.cpp:188] take(): Semaphore taken:  name: OpenEvt (0x3ffdd298), owner: connect
[ 10698][V][FreeRTOS.cpp:85] timedWait(): >> wait: Semaphore waiting: name: OpenEvt (0x3ffdd298), owner: connect for connect
[ 11151][D][BLEDevice.cpp:138] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... ESP_GATTC_CONNECT_EVT
[ 11162][V][BLEUtils.cpp:1202] dumpGattClientEvent(): GATT Event: ESP_GATTC_CONNECT_EVT
[ 11172][V][BLEUtils.cpp:1229] dumpGattClientEvent(): [conn_id: 0, remote_bda: d3:07:19:xx:xx:xx]
[ 11181][D][BLEClient.cpp:185] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... ESP_GATTC_CONNECT_EVT
[ 11192][D][BLEDevice.cpp:138] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... ESP_GATTC_OPEN_EVT
[ 11202][V][BLEUtils.cpp:1202] dumpGattClientEvent(): GATT Event: ESP_GATTC_OPEN_EVT
[ 11210][V][BLEUtils.cpp:1315] dumpGattClientEvent(): [status: ESP_GATT_OK, conn_id: 0, remote_bda: d3:07:19:xx:xx:xx, mtu: 23]
[ 11221][D][BLEClient.cpp:185] gattClientEventHandler(): gattClientEventHandler [esp_gatt_if: 4] ... ESP_GATTC_OPEN_EVT
[ 11232][V][FreeRTOS.cpp:136] give(): Semaphore giving: name: OpenEvt (0x3ffdd298), owner: connect
[ 11242][V][FreeRTOS.cpp:105] timedWait(): << wait: Semaphore name: OpenEvt (0x3ffdd298), owner: <N/A> released: 1
[ 11245][V][BLEUtils.cpp:967] dumpGapEvent(): Received a GAP event: ESP_GAP_BLE_AUTH_CMPL_EVT
[ 11260][V][BLEClient.cpp:162] connect(): << connect(), rc=1
[ 11260][V][BLEUtils.cpp:1021] dumpGapEvent(): [bd_addr: d3:07:19:xx:xx:xx, key_present: 0, key: ***, key_type: 0, success: 1, fail_reason: 0, addr_type: ***, dev_type: ESP_BT_DEVICE_TYPE_BLE]
[ 11283][I][esp32bridge.ino:149] writeNewUnlockKey(): [ESP32OmBlePy] get service
[ 11283][I][BLEDevice.cpp:231] gapEventHandler(): ESP_GAP_BLE_AUTH_CMPL_EVT
...

Maybe you can try verbose logging,
or just trying other options for the bonding like ESP_LE_AUTH_NO_BOND for securityP->setAuthenticationMode.

[gotnone]

I was running into the same issue with an OMRON EVOLV.

I was able to get a little bit further by adjusting the connect address_type to BLE_ADDR_TYPE_PUBLIC.

omblepy/esp32bridge/esp32bridge.ino

Line 258 in 406f706

clientP->connect(macAdderss, BLE_ADDR_TYPE_RANDOM);

omblepy/esp32bridge/esp32bridge.ino

Line 271 in 406f706

clientP->connect(macAdderss, BLE_ADDR_TYPE_RANDOM);

I don't know a lot about this address_type, and what it means to change to BLE_ADDR_TYPE_PUBLIC. I modified the scan callback to print out the address_type and it gives back an address_type of 0, which matches this enum value. FYI the BLE_ADDR_TYPE_RANDOM value is 1. I did see some other nearby devices that claimed to have address_types 1, as well as a few others with 0.

"mac": "34:ff:xx:xx:xx:xx", "address_type": 0, "name": "BLEsmart_##########################", "rssi": -70
"mac": "4b:xx:xx:xx:xx:xx", "address_type": 1, "name": "", "rssi": -50
...

I am currently diagnosing some errors that probably should be warnings espressif/arduino-esp32#3869 (comment)

[ 17256][E][BLERemoteCharacteristic.cpp:289] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
[ 17266][E][BLERemoteCharacteristic.cpp:289] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
[ 17276][E][BLERemoteCharacteristic.cpp:289] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
[ 17286][E][BLERemoteCharacteristic.cpp:289] retrieveDescriptors(): esp_ble_gattc_get_all_descr: Unknown
p OK

[userx14]

Hi @gotnone ,

really interesting, I think the BLE address randomization is used as a privacy feature. Something like bluetooth tracking beacons or Android phones will use randomized addresses to avoid being easily tracked.

For the omron devices, they have their mac-address printed on a sticker on the case and I have never seen them change, so it has to be a fixed manufacturer assigned mac address that they use for advertising. Your code is probably correct for all omron devices.

I don't remember the details, but likely I reused some arduino code from another project with environment sensors and missed that this needs a change.

The other messages you are getting could just be warnings, did you test reading the records already?

Best regards,
Benjamin

[gotnone]

@userx14 Thanks for the additional info.

The messages I was getting regarding the missing gatt characteristic descriptors are essentially warnings, but are being printed because I overlooked the instructions (4) to set -DCORE_DEBUG_LEVEL=0. I don't have the arduino IDE installed and had imported the project into platformio. Either platformio or the Arduino library must default to printing Error Level messages. Unfortunately the BLE Library code logs the missing descriptor message at Error Level.

https://github.com/espressif/arduino-esp32/blob/422e52684b824a3fde85989ea90abf6624f2b0eb/libraries/BLE/src/BLERemoteCharacteristic.cpp#L286-L289

After getting the Error messages to go away I was able to test both pairing and data transfer successfully.

I did learn that once the pairing is created with the Omron device it does not seem to matter which address type is selected.

omblepy/esp32bridge/esp32bridge.ino

Line 271 in 406f706

clientP->connect(macAdderss, BLE_ADDR_TYPE_RANDOM);

However, if you change the mac address of the ESP32 the connect function requires the address_type field to be BLE_ADDR_TYPE_PUBLIC in order to pair with this new mac address.