Some beacons jumping a lot between working and "Unknown"

[eigenphase]

iPhones, Apple watches (Private BLE Device) as well as Tesla key seems to work well and predictably.

But now I got three Blue Charm beacons to track the locations of my trash can and they constantly jump between found and "Unknown".

All three trackers are next to each other (on the trashcans). I have a total of 5 ESPhomes with bluetooth proxy.

One device that bothers me the most is "hpwh", it is the closest and shows around 40ft most of the time but shortly after goes to "Unknown".

I have tried increasing the Devtracker timeout to 120s and increasing the Bluecharm's output power to 8dBm ... nothing changed in terms of reliability.

I understand that there are weak signals and varying RSSI values wouldn't surprise me as much, but what is really causing the BLE proxies just jumping to "Unknown" all the time?

Here's an example

In this example you can see that Trashcan Blue is "found" by hpwh which is the closest one but Trashcan Black and Trashcan Green are both as "Unknown". Trashcan Black has instead "vue-utility" which is really far away.

A few seconds later everything will randomly flip again. Note that the trashcans are right next to each other.

[WolfeCub]

According to the wiki (link here):

Devtracker Timeout in seconds to consider a device as "Not Home" applies only to the device_tracker platform.

I'm curious if there's another setting. I also have an issue where my wife's android phone is very flaky. The area sensor looks like this:

Whereas my fitness band is rock solid.

[eigenphase]

Also adding now:

As can be seen, iPhone and watch all stable, Tesla too (just not shown because it's "Away").
All three trashcans (Bluecharm BLE beacons) are jumping around wildly

[agittins]

@eigenphase could you attach the result of a "download diagnostics"? I know others have had some issues with bluecharms having long intervals between advertisements, but that would normally cause "unknown" rather than area switches.

Do you have any bluetooth devices that an HA integration might be actively controlling (things like blinds, curtains etc) that might be causing some proxies to be spending their time connecting out instead of scanning?

@WolfeCub do you have the iBeacon transmitter enabled on your Wife's HA Companion App? This tends to give a reliable signal, while without it you might not get regular adverts. Also in the HA app the default "ultra low" power seems to be too low for most situations, and medium power might give much more reliable detection. Failing that, can you also attach a diagnostics so I can see what's going on under the hood?

A workaround that you might want to consider is using the "last seen" sensor - it's disabled by default, but if you enable it that will give you a reliable sensor for area that doesn't time out when it loses contact. This might be suitable for bins, but less so for a phone, for example.

[WolfeCub]

do you have the iBeacon transmitter enabled on your Wife's HA Companion App

I do actually but found it gave very similar results.

This was at low power though so I can give medium a try. Would you say that the companion app is preferable to the private BLE?

Is there a straightforward way to see if it's leaving the range of the room vs outside the range of detection? I have my rooms relatively small for use with my fitness tracker but I'm wondering if the phone strength is just significantly weaker? I did try drastically increasing the room size and the results were the same but I'd like to rule it out entirely.

I have been using the "last seen" in the mean time but I was hoping to figure out a way to get the phone more reliable.

I appreciate you taking the time to look. I'll hold off on having you look through the debug output until I can debug a little further and test the higher signal strength.

[mdklapwijk]

Do you have any bluetooth devices that an HA integration might be actively controlling (things like blinds, curtains etc) that might be causing some proxies to be spending their time connecting out instead of scanning?

This what @agittins said; I also had issues with devices going to 'unknown' (very) frequently. Multiple calibrations with different devices did not help, devices still went 'unknown' even when only 10cm apart.

After switching the Shelly Dimmer Gen3's from active to passive 'Bluetooth scanner mode' the issue seems more or less resolved:

• Sony Xperia 5 III, did not jump scanner, nor listed unknown for 25+ minutes at 2.4m distance.
• Shelly BLU Button 1, jumped between 2 scanners once, never listed unknown for 25+ minutes at 1.0m and 4.5m distances.

[WolfeCub]

@mdklapwijk What setup do you have on the Sony Xperia? Companion app? Private BLE?

[mdklapwijk]

@mdklapwijk What setup do you have on the Sony Xperia? Companion app? Private BLE?

Companion app, 10Hz and high power, since I had issues with it hopping rooms/scanners and going 'unknown'. Will dial it back if everything stays stable now.

I now also switched my esphome proxies to passive, which seems to work correctly:

esp32_ble_tracker:
  scan_parameters:
    active: false

This should also prevent additional power drain of battery powered devices (#657), if I read the docs correctly:

active (Optional, boolean): Whether to actively send scan requests to request more data after having received an advertising packet. With some devices this is necessary to receive all data, but also drains those devices’ power a bit more. Some devices don’t need this, in that case you can save power and RF pollution by setting it to false. Defaults to true.

[mdklapwijk]

Still some scanner hopping and occasional 'unknown', but that seems to be due to wifi coverage and the Shelly's automatic roaming settings; now limiting the iot network to the 3 strongest ap's in the network and monitor the rssi on the Shelly's.

[kaptain79]

I believe I am having similar issues.
I dipped my toes into this a month ago and had 6 ESP32's installed around the house to track a blue charm beacon on my dog. He was everywhere! Reasearch told me it was the switching from BT to WIFI that was causing the inconsistency. I removed all the ESP32's and currently have 2 WT32-ETH01's, master bedroom and living room. My dog sleeps at the foot of my bed so this morning I was expecting the log to show master bedroom all night. Instead, I see that it goes "unknown", 8 seconds later living room, 10 seconds back to master bedroom and repeat. The time before it drops off to unknown is random. Just checked, my watch very similar, so that tells me it's the proxies.

esp32_ble_tracker:
bluetooth_proxy:

This is what I have in my configs. Any suggestions that have worked for you? I am going to add
scan_parameters:
active: false
to my Livingroom sensor and see if it makes any difference.

[kaptain79]

I was able to resolve my own problem.

First thing I tried was increasing the radius back to 20M, I had it set to 5M thinking that's how far out the proxies would look. I thought I was creating dead space between the 2 proxies, in turn creating all the jumps. This did not help.

Next, I tried setting the active scanning parameters to false, again nothing.

Finally, I read somewhere about interval and window using active scanning. I could be wrong here: default scan window is 30ms and interval is 320ms. After a 30ms scan it waits 320ms before scanning again. I believe the short window is the cause of the flapping, not enough time for the radio to see my BT device. I was about to try the suggested 120ms window/320ms interval, but I saw a note that proxies on ethernet have good success using a 1:1 ratio (1000ms/1000ms). That is what I tried, and it is working great.

My code is below, this is for ethernet, not wifi devices.

---- Ethernet (WT32-ETH01 uses LAN8720 PHY) ----

ethernet:
type: LAN8720
mdc_pin: GPIO23
mdio_pin: GPIO18
clk:
pin: GPIO0
mode: CLK_EXT_IN
phy_addr: 1
power_pin: GPIO16 # if link doesn't come up, try removing this line

---- Bluetooth Proxy ----

esp32_ble_tracker:
scan_parameters:
active: True # Whether to send scan-request packets to devices to gather more info (like devicename)
interval: 1000ms # suggested 211ms # default 320ms
window: 1000ms # suggested 120ms # default 30ms
bluetooth_proxy:
active: True

[WolfeCub]

I upped the companion app Bluetooth transmission to "High" power. It still seems to go unavailable a fair amount as well as bouncing to a few other rooms briefly even when completely stationary.

[eigenphase]

I am not sure why @kaptain79 's proposal of "interval: 1000ms # suggested 211ms # default 320ms
window: 1000ms # suggested 120ms # default 30ms" works better, it's counter intuitive.

For me, I could fix issues by leaving the BLE proxy note settings unchanged (it says changing them may have big impact on stability when used together with Wifi), however, the beacons I changed the advertisement interval to 100ms. Now things work for me but also at 10x energy consumption.

Here is why: 320ms/30ms means that scanning repeats every 320ms (~3x/second) and in each scanning period only 30ms are used to actually "listen" (capture data). In other words, only about 10% of the time the BLE proxy can actually receive an advertisement.

Now you can imagine that if the advertisement interval of your beacons is high (1s or even higher), it will take many attempts until one hits the 30ms window. Decreasing the advertisement interval substantially increased the chance that one is successfully received but it comes at higher energy cost (scales roughly linear).

I believe the internal timeout is like 30s, which means, if no advertisement for an iBeacon is received for 30s, it goes to "Unknown".

Maybe @agittins can confirm this, and, at some point I wanted to add a setting to change this timeout value, maybe even per-tracker. For example, It should be low for my iPhones and watches but for my beacons on the trashcan, 5min would be plenty and I could crank up the advertisement interval again.

[WolfeCub]

@eigenphase I've been running 320ms/300ms and I've still noticed the one beacon (android phone) jumping to unknown a lot.