Successfully migrated from ZHA to Z2M without re-pairing #24478
Replies: 20 comments 55 replies
-
|
Interesting! Had a chat with @Nerivec about this, it should be easy to add an automatic interview when an unknown device sends a message. So the procedure would be:
|
Beta Was this translation helpful? Give feedback.
-
Yes. Want me to write a function which reads the relevant ZHA settings and returns them?
That's right. No more pair-crawling :D |
Beta Was this translation helpful? Give feedback.
-
Yes, would be cooler if it could also generate a part of the z2m |
Beta Was this translation helpful? Give feedback.
-
That's the stuff from the coordinator_backup.json, so, should already be fine on that point. Just need to copy it over from ZHA (I think you can even download it from ZHA?). The newer coordinator_backup.json format (once finalized) should have an independent list of ieee/nwk pairs, which would allow to fill in the database with the minimal info, from Controller, if the result of adapter start was Just a quick thought, but something along these lines should be doable. |
Beta Was this translation helpful? Give feedback.
-
|
In my case the downloaded As I understand, all of this is already on the radio so you should be able to just stop ZHA, start Z2M and point it at the same radio? As @thargy alludes to in this discussion. |
Beta Was this translation helpful? Give feedback.
-
|
Only the IEEE and Network address is held on each radio to confirm it is registered with the network. The device's manufacturer ID, model, clusters, etc. can be interviewed. What isn't available is a friendly name/entity ID, so either that data needs to be grabbed from ZHA's config, or supplied manually. There's an argument for Z2m to populate the IEEEAddress as an attribute against the entity in HA as it would be useful for restoring an association. I've also asked about populating a list of child entities for groups, or at least supply a GroupID. |
Beta Was this translation helpful? Give feedback.
-
|
What to do after coming home from a nice party? Feed the cats and deliver the code, of course! Note: The most straightforward way would be to instruct users to download a backup from ZHA and then disable it. However, you could probably also just curl the backup from the other extension.
|
Beta Was this translation helpful? Give feedback.
-
|
Note you only need the IEEE and Network addresses. You can set the "type" to "Unknown". The interview will grab the correct type and everything else (including the Manufacturer ID). For coordinator's that support backups, the IEEE and Network addresses should be retrieved from the coordinator as that is the definitive source of truth. For a ZHA migration, you only need to grab the IEEE and friendly name mapping. I believe all the relevant state should also be queried from the device. Most importantly, you have to batch the interviews otherwise you are likely to crash the network/coordinator. In my testing a maximum of 3/4 concurrent interviews is safe. |
Beta Was this translation helpful? Give feedback.
-
|
Does |
Beta Was this translation helpful? Give feedback.
-
|
Z2M, thanks for catching this! We want it to be stopped before editing the database on disk. |
Beta Was this translation helpful? Give feedback.
-
|
I was able to download zigbee.db on a HAOS OVA install by using the File editor addon. Then dumping some attributes from it there are quite a few tables and they are versioned: I think we're only interested in There's not a good way of exporting devices from HA and I don't have access to |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for this. Note that the ext_pan_id is in reverse order in the Z2M configuration vs. a ZHA backup for me (no clue who is right) ZHA backup "network_info": {
"extended_pan_id": "7e:51:d3:4f:64:50:5a:6e",Z2M config ext_pan_id:
- 110
- 90
- 80
- 100
- 79
- 211
- 81
- 126 |
Beta Was this translation helpful? Give feedback.
-
|
Big-endian, little-endian, it's not a matter of "right", but "right where it's used". Z2M adapts it as necessary throughout the code, so does ZHA. 😉 |
Beta Was this translation helpful? Give feedback.
-
|
They might, but my coordinator (yellow) was not happy to see a different ext_pan_id. Something in the like of "was configured with", with the ZHA ext_pan_id in reverse order if I specify the ext_pan_id in config the same way as ZHA. |
Beta Was this translation helpful? Give feedback.
-
|
ZHA stores them one way, and Z2M the other, so you indeed need to reverse them when switching from one to the other, in the file that is. The various systems in Z2M usually take care of this conversion when importing a coordinator_backup.json into Z2M, but the process here is a bit more "hands-on", so some conversions have to be made manually. |
Beta Was this translation helpful? Give feedback.
-
Hey. First of all, thank you for all your contributions. I'm also in the "ZHA continues its progress down the drain, and I don't want my network to sink with the ship" boat. I was thinking about the "hands-on way" migration this weekend, but after reading your post, maybe I should postpone it for some time, if not only to make things easier for myself, then to provide a "it worked/it borked" datapoint on the less "hands-on" approach. Question is, in your estimation, how much longer would I have to suffer ZHA restarting itself/doing other unwanted things before even a beta-quality procedure is implemented? ;) PS. Looking at things, I think implementing such procedure will result in influx of new Z2M users :> |
Beta Was this translation helpful? Give feedback.
-
|
ZHA currently does not follow the open coordinator backup format, which makes it hard to derive a proper logic from it (would have to walk every single version from the moment they deviated from the format to adapt to every change, a monstruous task). |
Beta Was this translation helpful? Give feedback.
-
|
Tried following this but something is unhappy: A quick web search suggests that maybe the problem is that there are other routers around that are still running the network and that's causing some kind of conflict? |
Beta Was this translation helpful? Give feedback.
-
|
Figured out the issue was the missing coordinator backup. Adding it got that error to go away. |
Beta Was this translation helpful? Give feedback.
-
|
I want to say thank you for this thread, and add some notes for anyone else who wants to try this:
So it took me a while to realize that was why it said this.. I originally assumed it was smarter than me, but none of my devices were talking right. Eventually I used the settings/advanced menu of z2m's front end to reverse the bytes and that fixed it (I assume deleting the
|
Beta Was this translation helpful? Give feedback.
-
|
Happy new year everyone. I'm starting from scratch on the zigbee2mqtt side. Provided Z2M configuration is complete (Zigbee network...), does it hurt to start without a device list so that I get the base |
Beta Was this translation helpful? Give feedback.
-
|
I was wrong: if there is no
|
Beta Was this translation helpful? Give feedback.
-
|
Hello @araemo. |
Beta Was this translation helpful? Give feedback.
-
|
Everything still needed to be interviewed, but according to this comment #24478 (reply in thread), you only need the network addr and ieee addr.
So while I did this: This should work the same: |
Beta Was this translation helpful? Give feedback.
-
|
Yep, I redid 200+ devices using just type ‘Unknown’ with IEEE and network address (and sequential ID of course). Not sure what @toine512 ’s problem was? A reinterview populated everything else. |
Beta Was this translation helpful? Give feedback.
-
That's what I did but with type: EndDevice. It works.
With type Unknown Z2M was ignoring all entries (explicit message in the log), no device shown in the Web UI. With type EndDevice it's ok. Status is unsupported until interview is performed. |
Beta Was this translation helpful? Give feedback.
-
|
So here's what I did. Based on details in pretty much all previous posts. I performed data manipulation on the side for ease of use, config files are copied back and forth with the machine running zigbee2mqtt.
Everything works. |
Beta Was this translation helpful? Give feedback.
-
|
I think I followed this writeup to a T as you have written it, but I get stuck at step 5. Seems Z2M is upset about migrating. Other solutions to this problem only suggest to generate new keys. But since I am trying to keep my network intact, I would like to avoid that as much as possible! |
Beta Was this translation helpful? Give feedback.
-
|
Did you reverse the order of bytes of As long as it doesn't work right, make sure to remove |
Beta Was this translation helpful? Give feedback.
-
Yep, exactly. I am not seeing coordinator_backup.json get created. However I am unsure I am looking in the right place. Are you using |
Beta Was this translation helpful? Give feedback.
-
Yes. |
Beta Was this translation helpful? Give feedback.
-
|
Can you help me how to setup pan, epan and network key in z2m config? this is my ZHA config: |
Beta Was this translation helpful? Give feedback.
-
|
Dear all, |
Beta Was this translation helpful? Give feedback.
-
|
OK, I'll look at this device type. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for pointing it out. Reading #24198, I found no changes to network parameters or the very simple device list that is generated. So it should be ok. I hope @rjDipcord's issue doesn't come from this. There is a massive |
Beta Was this translation helpful? Give feedback.
-
|
Not good... I have this error when starting Z2M v2 after the update migrate the configuration.yaml. In configuration.yaml I have my devices as: |
Beta Was this translation helpful? Give feedback.
-
|
All good now, for an unknown reason the last 2 devices where missing the leading "0x" once migrated. |
Beta Was this translation helpful? Give feedback.
-
|
@toine512 So I was able to get the sensor history back. I have a device called "congelateur" which is this one https://www.zigbee2mqtt.io/devices/TS011F_plug_1.html#tuya-ts011f_plug_1 1 . Delete the entity sensor.congelateur_consommation, home assistant did not purge the data 
Now, if I want to keep Z2M naming convention for future devices like this one:
|
Beta Was this translation helpful? Give feedback.
-
|
Would someone be able to write a (detailed) howto on how to do this? I've started of with ZHA but feel this is a mistake:
You would have everlasting ❤️ 🙏 . |
Beta Was this translation helpful? Give feedback.
-
|
Toine512 made an almost, almost perfect manual. Thanks Antoni! I managed to migrate my network without re-pairing (And thank you XX seidtgeist your idea of starting this post) |
Beta Was this translation helpful? Give feedback.
-
|
I believe this should work? Using the Home Assistant Terminal and Studio Code Server add-ons (for easier editing/access), in Terminal run: $ tail -n 1 config/zigbee2mqtt/database.db
# note the highest ID value, replace below where it says "+ 2"
$ sqlite3 config/zigbee.db
sqlite> select CONCAT('{"id": ', (ROW_NUMBER() OVER (ORDER BY d.ieee)) + 2, ', "type": "EndDevice", "ieeeAddr": "0x', REPLACE(d.ieee, ':', ''), '", "nwkAddr": ', d.nwk, '}') FROM devices d JOIN endpoints e ON d.ieee = e.ieee ;This will give you a list like {"id": 8, "type": "EndDevice", "ieeeAddr": "0x00178801023dd160", "nwkAddr": 23188}That you should be able to paste into the zigbee2mqtt/database.db. |
Beta Was this translation helpful? Give feedback.
-
|
Also note that you still have to manually copy, reformat and potentially reverse PAN, ePAN, network key etc. those are visible in the ZHA config or the zigbee.db networks config. |
Beta Was this translation helpful? Give feedback.
-
|
I have my first device up & running with Z2M but after reading this thread and owning the SLZB-06M, I'm quite scared to be honest to make the move. ZHA has been running stable here but it does not support all the devices that I have. Anybody else here has the SLZB-06M? |
Beta Was this translation helpful? Give feedback.
-
|
by they way can anyojne help with correct reformating of PAN, ePAN and network key? my devices not showing up for me |
Beta Was this translation helpful? Give feedback.
-
|
@toine512 @seidtgeist Thx! I prepared some page to do all that stuff :) https://paweb88.github.io/zha-z2m/index.html ( soryy is in Polish right now, but I plan to translate it and make some UI ) |
Beta Was this translation helpful? Give feedback.
-
|
That worked for me like a charm! |
Beta Was this translation helpful? Give feedback.
-
|
This is really cool. I'm going to give this a whirl. I wrote a quick python script which can run under pyscript to generate the yaml and json files. Doesn't require a backup file and interfaces with HA API to gather all info. https://gist.github.com/archite/1930050f3c5ac032b418f83313d8d5dc Once again, thanks for finding the process! |
Beta Was this translation helpful? Give feedback.
-
Thank you, that worked! |
Beta Was this translation helpful? Give feedback.
-
Since I always have been struggling a bit, here are step-by-step instructions that made it working for me (I use HAOS in a virtual machine):
Done. |
Beta Was this translation helpful? Give feedback.
-
|
I added some points to the above description that I had to search around this thread for in order to do them - So I added them. Some things were quite confusing having never used z2m before. 0a. Leave ZHA enabled for now
Feel free to add this to the first post. Except 11 everything can be found in this thread. And I think this covers everything needed for first time users of z2m. Maybe watch a short video tutorial on how to install z2m first as that may help a lot. |
Beta Was this translation helpful? Give feedback.
-
|
@extremecarver Thank you for the excellent summary. Just migrated my environment with the help of this post.
After that Z2M starts without problems, but my devices were still missing. Unfortunately it seems auto-interviewing existing devices via MQTT does not work... |
Beta Was this translation helpful? Give feedback.
-
|
hmm, so you got stuck similarily. The worse thing for me was that any devices needed to do all the actions that they can do before I can select them in automations. So that caused me a lot of confusion. I guess using the site is a bit easier than the script because there is no need to reformat? But great it was of help - it took me really long figuring out many of those steps and order. And yeah bad enough after the interview that you need to push all buttons/things to make it work actually. That's something really missing here as it cannot be assumed people know z2m peculiar decisions. If it weren't for some curtain switches which I cannot learn with ZHA - I think for me sticking with ZHA would have been the better choice as it took loads of time to redo all my automations that broke. |
Beta Was this translation helpful? Give feedback.
-
|
You must ensure that you’ve added this block to your |
Beta Was this translation helpful? Give feedback.
-
|
Hi! |
Beta Was this translation helpful? Give feedback.
-
|
why is it skipping [2025-06-16 12:19:46] debug: zh:controller: Data is from unknown device with address '63414', skipping... |
Beta Was this translation helpful? Give feedback.
-
|
found the issue!! Google translate f***cked up the the webpage so my config file was bad. Issue solved. |
Beta Was this translation helpful? Give feedback.
-
|
Well pasting in the configuration after installation but before onbarding I end up with bad gateway - if I don't pase anything in it works..
|
Beta Was this translation helpful? Give feedback.
-
|
If I don't copy in the pan ID and ext pan ID it does work. I read somewhere that moving from zha to z2m maybe only works with Texas Instruments sticks? Is there a problem with my Sonoff dongle? But then of course I have to start from scratch which is not funny.. |
Beta Was this translation helpful? Give feedback.
-
|
well solved this by finding some old post about using a different channel first. So everything as is not booting, set channel 24 booted up. Stopped it. Went into backup and config and set back to channel 25 and it booted. Still cannot find any devices even though the configuration is correct now... Will next try switching off power for the full flat to hope to take down any routers and hope it works afterwards. |
Beta Was this translation helpful? Give feedback.
-
|
Not sure what's going on here after importing the database: [18:44:30] INFO: Preparing to start... |
Beta Was this translation helpful? Give feedback.
-
|
Oh well - I forgot the last step need to press pairing button on all battery powered devices for interview. Mains powered the interview button alon and then using the device/switch/button with an action was enough. So the not supported is normal. |
Beta Was this translation helpful? Give feedback.
-
|
After thoroughly reading this thread, looking at all the code and doing a mental calculation of the time/work involved, I figure the processes in here are worth doing only if you have over 100 Zigbee devices. Less than that and it seems like it will be much faster and much less headache to simply re-pair all of them manually, including making any adjustments to automations. |
Beta Was this translation helpful? Give feedback.
-
|
It's also worth noting that the limit is 200 devices and, from extensive real world testing, I recommend creating a new network every 70-100 devices. That way you go from frequent issues, to bomb proof. |
Beta Was this translation helpful? Give feedback.
-
|
I got this working with a couple scripts. General Steps
NOTE: Make sure Z2M is stopped while you are changing the database file, as it overrides/resets it on shut down. Overview of necessary Z2M config/config/zigbee2mqtt/configuration.yamladvanced:
channel: 25
network_key:
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
pan_id: 0
ext_pan_id:
- 0
- 0
- 0
- 0
- 0
- 0
- 0
- 0
devices:
'0x0000000000000000':
friendly_name: SLZB-06M Zigbee Cooordinator
'0x000000000000000a':
friendly_name: Dining Room Sliding Door Sensor
'0x000000000000000b':
friendly_name: Front Door Sensor
groups:
'1':
friendly_name: Office Wall Lights
off_state: last_member_state
'2':
friendly_name: Office Ceiling Lights/config/zigbee2mqtt/database.db{"id":1,"type":"Coordinator","ieeeAddr":"0x0000000000000000","nwkAddr":0, ... }
{"id":2,"type":"Unknown","ieeeAddr":"0x000000000000000a","nwkAddr":00000}
{"id":3,"type":"Unknown","ieeeAddr":"0x000000000000000b","nwkAddr": 00000}
...
{"id":100,"type":"Group","groupID":1,"members":[{"deviceIeeeAddr": "0x000000000000000b", "endpointID": 11}],"meta":{}}
{"id":101,"type":"Group","groupID":2,"members":[],"meta":{}}Scripts1. Get device and group namesNOTE: You maybe have multiple devices with the same name, ZHA allows this but Z2M does not. Update any duplicate names after you copy over to NOTE: Devices bound to the group endpoints (like switches), don't seem to pick up those definitions in the Z2M UI for the device. Everything will still work, but you won't be able to see the switch is controlling the group in Z2M. Not sure how to pull this in without re-binding the switch to the group. NOTE: Defining the groups in the Z2M config file, and database file should create the groups, but it doesn't. I ended up manually having to define the groups in the UI with the appropriate group IDs, and then could copy over the devices for the groups into the database afterwards. 🤷 TODO: ZHA defines a default "Lightlink" group with Group ID 0. I removed this from my config/database. Should probably automate removing this. Create this python script in `/config`, run it, and it will output a `z2m_devices.yaml` and a `z2m_groups.yaml` file.import json
import sys
import collections
# CONFIGURATION
# IMPORTANT: This script assumes it's running from `/config` and that `/config/.storage/device_registry` exists.
registry_file = '.storage/core.device_registry'
output_file = 'devices.yaml'
def format_ieee(zha_id):
"""
Converts ZHA '00:15:8d:00:07:e0:67:8f' to Z2M '0x00158d0007e0678f'
"""
clean = zha_id.replace(":", "").lower()
return f"0x{clean}"
def run_name_extraction():
try:
with open(registry_file, 'r', encoding='utf-8') as f:
data = json.load(f)
devices = data['data']['devices']
z2m_devices_raw = {}
# Dictionary to track the count of each friendly name
name_counts = collections.Counter()
print(f"Scanning {len(devices)} registry entries...")
# --- PASS 1: Extract names and count duplicates ---
for dev in devices:
# Only process ZHA devices
zha_id = None
for id_pair in dev.get('identifiers', []):
if id_pair[0] == 'zha':
zha_id = id_pair[1]
break
if zha_id:
# Determine the best name (User > HA Name > Manufacturer/Model)
friendly_name = dev.get('name_by_user') or dev.get('name')
if not friendly_name:
friendly_name = f"{dev.get('manufacturer', 'Unknown')} {dev.get('model', 'Device')}"
# Store the raw mapping
ieee = format_ieee(zha_id)
z2m_devices_raw[ieee] = friendly_name
# Count the name for later duplicate checking
name_counts[friendly_name] += 1
# --- PASS 2: Build final output and check for duplicates ---
final_z2m_output = {}
duplicate_warnings = []
zigbee_device_count = 0
for ieee, name in z2m_devices_raw.items():
zigbee_device_count += 1
# Check if this name is a duplicate
if name_counts[name] > 1:
duplicate_warnings.append((name, ieee))
# Use the original name for the final output, relying on the user
# to correct the duplicates listed in the warnings.
final_z2m_output[ieee] = {"friendly_name": name}
# --- Output Generation ---
print(f"\nFound {zigbee_device_count} Zigbee devices. Writing to {output_file}...")
# Output to YAML format
yaml_content = "devices:\n"
for ieee, details in final_z2m_output.items():
# Use single quotes around the friendly_name for safety with special characters
yaml_content += f" '{ieee}':\n"
yaml_content += f" friendly_name: '{details['friendly_name'].replace('\'', '\'\'')}'\n" # Escape single quotes
# Write YAML file
with open(output_file, 'w', encoding='utf-8') as f:
f.write(yaml_content)
print("\n" + "="*50)
print("✅ SUCCESS: devices.yaml content is ready.")
print("="*50 + "\n")
# --- Duplicate Warnings ---
if duplicate_warnings:
print("⚠️ WARNING: DUPLICATE FRIENDLY NAMES DETECTED ⚠️")
print("Zigbee2MQTT does not support duplicate names. You MUST manually update")
print("the names listed below in your newly created 'devices.yaml' file.")
print("\nSuggested fixes: Append a room or number (e.g., 'Hallway Sensor 1', 'Hallway Sensor 2').")
print("-" * 40)
# Get a list of unique duplicate names for cleaner output
names_to_fix = set([item[0] for item in duplicate_warnings])
for name in names_to_fix:
print(f"Name: '{name}' is duplicated on the following device(s):")
# List all devices that have this duplicate name
for dup_name, ieee in duplicate_warnings:
if dup_name == name:
print(f" - IEEE Address: {ieee}")
print("-" * 40)
print("After editing 'devices.yaml', include it in your Z2M configuration.yaml.")
except FileNotFoundError:
print(f"❌ Error: File not found at '{registry_file}'.")
print("Please ensure you have copied the 'core.device_registry' file into the same directory as this script.")
sys.exit(1)
except json.JSONDecodeError:
print(f"❌ Error: Failed to parse JSON in '{registry_file}'. The file may be corrupted.")
sys.exit(1)
except Exception as e:
print(f"❌ An unexpected error occurred: {e}")
sys.exit(1)
# Run the main function
run_name_extraction()Replace or add these values to the top-level 2. Get your zigbee device addressesNOTE: This assumes you have a fresh Z2M server with just the coordinator defined (this will generate ids starting at Create this python script in `/config`, run it, and it will output a `z2m_device_entries.txt` and `z2m_group_entries.txt`.import json
import sqlite3
import sys
# CONFIGURATION
# IMPORTANT: This script assumes it's running from `/config` and that `zigbee.db` exists.
zigbee_db = 'zigbee.db'
devices_output_file = 'z2m_device_entries.txt'
groups_output_file = 'z2m_group_entries.txt'
def format_ieee(zha_id):
"""
Converts ZHA IEEE address to Z2M format '0x00158d0007e0678f'
"""
if isinstance(zha_id, str):
clean = zha_id.replace(":", "").lower()
if not clean.startswith('0x'):
return '0x' + clean
return clean
if isinstance(zha_id, bytes):
return '0x' + zha_id.hex()
if isinstance(zha_id, int):
return '0x{:016x}'.format(zha_id)
return str(zha_id)
def extract_devices_from_db():
"""Extract device information from ZHA database for database.db"""
try:
conn = sqlite3.connect(zigbee_db)
cursor = conn.cursor()
cursor.execute("SELECT ieee, nwk FROM devices_v13")
devices = cursor.fetchall()
conn.close()
z2m_devices = []
id_counter = 2 # Start at 2 (coordinator is 1)
for ieee_raw, nwk in devices:
# Skip the coordinator (usually nwk 0)
if nwk == 0:
continue
ieee_hex = format_ieee(ieee_raw)
entry = {
"id": id_counter,
"type": "Unknown", # Z2M will auto-correct during interview
"ieeeAddr": ieee_hex,
"nwkAddr": nwk
}
z2m_devices.append(entry)
id_counter += 1
return z2m_devices
except sqlite3.Error as e:
print(f"❌ Error accessing SQLite database '{zigbee_db}': {e}")
sys.exit(1)
def extract_groups_and_members():
"""Extract group information and memberships from ZHA SQLite database"""
try:
conn = sqlite3.connect(zigbee_db)
cursor = conn.cursor()
# Get groups
cursor.execute("SELECT group_id, name FROM groups_v13 ORDER BY group_id")
groups_data = cursor.fetchall()
# Get group memberships
cursor.execute("SELECT group_id, ieee, endpoint_id FROM group_members_v13")
memberships_data = cursor.fetchall()
conn.close()
# Build groups dictionary with members
groups = {}
for group_id, name in groups_data:
groups[group_id] = {
"name": name,
"members": []
}
# Add members to groups
for group_id, ieee_raw, endpoint_id in memberships_data:
if group_id in groups:
ieee_hex = format_ieee(ieee_raw)
groups[group_id]["members"].append({
"deviceIeeeAddr": ieee_hex,
"endpointID": endpoint_id
})
return groups
except sqlite3.Error as e:
print(f"❌ Error accessing groups in database: {e}")
sys.exit(1)
def write_device_entries(devices, filename):
"""Write device entries for Z2M database.db"""
with open(filename, 'w', encoding='utf-8') as f:
for device in devices:
json_line = json.dumps(device)
f.write(json_line + '\n')
def write_group_entries(groups, filename, start_id):
"""Write group entries for Z2M database.db"""
with open(filename, 'w', encoding='utf-8') as f:
id_counter = start_id
for group_id, group_data in groups.items():
entry = {
"id": id_counter,
"type": "Group",
"groupID": group_id,
"members": group_data["members"],
"meta": {}
}
json_line = json.dumps(entry)
f.write(json_line + '\n')
id_counter += 1
def main():
print("="*70)
print("ZHA to Zigbee2MQTT Database Migration Tool")
print("="*70 + "\n")
# Extract devices for database.db
print("📱 Extracting device addresses from ZHA database...")
db_devices = extract_devices_from_db()
print(f"✅ Found {len(db_devices)} devices\n")
# Extract groups and memberships
print("👥 Extracting groups and memberships from ZHA database...")
groups_with_members = extract_groups_and_members()
print(f"✅ Found {len(groups_with_members)} groups with memberships\n")
# Write device entries for database.db
write_device_entries(db_devices, devices_output_file)
print(f"✅ Device entries written to '{devices_output_file}'")
# Write group entries for database.db
# Start group IDs after devices (coordinator=1 + devices)
group_start_id = len(db_devices) + 2
write_group_entries(groups_with_members, groups_output_file, group_start_id)
print(f"✅ Group entries written to '{groups_output_file}'")
print("\n" + "="*70)
print("✅ SUCCESS: Migration files are ready!")
print("="*70 + "\n")
print("📋 Next steps:")
print(f"\n1. Open /config/zigbee2mqtt/database.db with VS Code:")
print(f" - CMD+Shift+P")
print(f" - 'Open File'")
print(f" - Enter '/config/zigbee2mqtt/database.db'")
print(f"\n2. Paste contents of '{devices_output_file}' into database.db")
print(f"\n3. Paste contents of '{groups_output_file}' into database.db")
if __name__ == "__main__":
main()Open
Paste the entries in there. 3. Get your coordinator keysCreate this script in `/config` and run it to get your coordinator keys:import sqlite3
import json
import sys
def extract_and_convert_zigbee_keys(db_path='zigbee.db'):
"""
Extract Zigbee coordinator keys from the database and convert to Z2M format.
"""
try:
# Connect to the database
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Query the backup JSON
cursor.execute("""
SELECT backup_json
FROM network_backups_v13
LIMIT 1
""")
result = cursor.fetchone()
conn.close()
if not result:
print("Error: No backup found in database.")
sys.exit(1)
# Parse the JSON
backup_data = json.loads(result[0])
network_info = backup_data.get('network_info', {})
# Extract values
network_key_octets = network_info.get('network_key', {}).get('key', '')
pan_id_hex = network_info.get('pan_id', '0000')
ext_pan_id_octets = network_info.get('extended_pan_id', '')
channel = network_info.get('channel')
# ~~~~~~~~~ Network Key (32 hex chars = 16 bytes) ~~~~~~~~~
clean_network_key = network_key_octets.replace(':', '').replace(' ', '').strip()
if len(clean_network_key) != 32:
print(f"Error: network_key must be exactly 32 hex characters (16 bytes). Found: {len(clean_network_key)} characters")
sys.exit(1)
network_key_bytes = []
for i in range(0, 32, 2):
hex_byte = clean_network_key[i:i+2]
decimal_byte = int(hex_byte, 16)
network_key_bytes.append(decimal_byte)
network_key_output = " network_key:"
for byte in network_key_bytes:
network_key_output += f"\n - {byte}"
# ~~~~~~~~~ PAN ID (4 hex chars) ~~~~~~~~~
clean_pan_id = pan_id_hex.strip().replace('0x', '').upper()
if len(clean_pan_id) != 4:
print(f"Error: pan_id must be exactly 4 hex characters. Found: {clean_pan_id}")
sys.exit(1)
pan_id_decimal = int(clean_pan_id, 16)
pan_id_output = f" pan_id: {pan_id_decimal}"
# ~~~~~~~~~ Extended PAN ID (16 hex chars = 8 bytes) ~~~~~~~~~
clean_ext_pan_id = ext_pan_id_octets.replace(':', '').replace(' ', '').strip()
if len(clean_ext_pan_id) != 16:
print(f"Error: extended_pan_id must be exactly 16 hex characters (8 bytes). Found: {len(clean_ext_pan_id)} characters")
sys.exit(1)
ext_pan_id_bytes = []
for i in range(0, 16, 2):
hex_byte = clean_ext_pan_id[i:i+2]
decimal_byte = int(hex_byte, 16)
ext_pan_id_bytes.append(decimal_byte)
ext_pan_id_output = " ext_pan_id:"
for byte in ext_pan_id_bytes:
ext_pan_id_output += f"\n - {byte}"
# ~~~~~~~~~ Channel ~~~~~~~~~
channel_output = f" channel: {channel}" if channel is not None else ""
# Print results
print()
if channel_output:
print(channel_output)
print(network_key_output)
print(pan_id_output)
print(ext_pan_id_output)
print("\n✅ Conversion Complete. Paste/replace these in the 'advanced:' section in configuration.yaml")
except sqlite3.Error as e:
print(f"Database error: {e}")
sys.exit(1)
except json.JSONDecodeError as e:
print(f"JSON parsing error: {e}")
sys.exit(1)
except Exception as e:
print(f"Unexpected error: {e}")
sys.exit(1)
if __name__ == "__main__":
# Allow optional database path argument
db_path = sys.argv[1] if len(sys.argv) > 1 else 'zigbee.db'
extract_and_convert_zigbee_keys(db_path)Replace the 4. Disable ZHA and enable Z2M5. Re-interview all devicesNOTE: Battery powered devices will likely need to be woken to be re-interviewed. You can manually click the interview button next to the devices, or open up the web console and paste this script:
I give up trying to make this script work perfectly, it should mostly work. /**
* Recursively finds all elements matching the selector, including in Shadow DOMs and Iframes.
*/
function findAllDeepElements(selector, rootNode = document) {
const results = [];
const traverse = node => {
if (node.nodeType === Node.ELEMENT_NODE && node.matches(selector)) {
results.push(node);
}
// Check iframe
if (node.nodeName === 'IFRAME') {
try {
const iframeDoc = node.contentDocument || node.contentWindow.document;
if (iframeDoc?.body) {
traverse(iframeDoc.documentElement || iframeDoc.body);
}
} catch (e) {
console.warn(`Skipping cross-origin iframe`);
}
}
// Check shadow DOM
if (node.shadowRoot) {
for (const child of node.shadowRoot.children) {
traverse(child);
}
}
// Check regular children
if (node.children) {
for (const child of node.children) {
traverse(child);
}
}
};
if (rootNode.nodeType === Node.DOCUMENT_NODE) {
traverse(rootNode.documentElement);
} else {
traverse(rootNode);
}
return results;
}
/**
* Finds the scrollable container (currently just returns html element).
*/
function findScrollContainer(element) {
let parent = element.parentElement;
while (parent) {
if (parent.localName === 'html') return parent;
parent = parent.parentElement;
}
return window;
}
// --------------------------------------------------------------------------
// MAIN EXECUTION
// --------------------------------------------------------------------------
function reinterviewFailedDevicesDeepScan() {
console.log("🚀 Starting deep-scan interview script...");
// Find the table container
const listContainers = findAllDeepElements('#all-devices tbody');
if (listContainers.length === 0) {
console.error("❌ Error: Could not find '#all-devices tbody'");
return;
}
const listContainer = listContainers[0];
const actionDocument = listContainer.ownerDocument;
const scrollElement = findScrollContainer(listContainer);
const isWindowScroll = scrollElement === window;
console.log(`✅ Found container. Scroll mode: ${isWindowScroll ? 'Window' : scrollElement.tagName}`);
// Configuration
const INTERVIEW_DELAY = 3000; // 3s between each interview
const MODAL_DELAY = 300; // Time for modal to appear
const SCROLL_SETTLE_DELAY = 2000; // 2s wait after scroll for virtual list to load
const SCROLL_STEP = (isWindowScroll ? window.innerHeight : scrollElement.clientHeight) * 0.9;
// State tracking
const processedIndexes = new Set();
let interviewInitiatedCount = 0;
let lastScrollHeight = isWindowScroll
? actionDocument.documentElement.scrollHeight
: scrollElement.scrollHeight;
/** Process all visible failed devices in the current view */
const processVisibleFailedDevices = () => {
let batchIndex = 0; // Local counter for this batch
listContainer.querySelectorAll('tr').forEach(row => {
const index = row.getAttribute('data-item-index');
if (processedIndexes.has(index) || index === '$0') return;
if (!row.querySelector('[title="Interview failed"]')) return;
processedIndexes.add(index);
setTimeout(() => {
const interviewBtn = row.querySelector('[data-tip="Interview"]');
if (!interviewBtn) {
console.warn(`No interview button for index ${index}`);
return;
}
interviewBtn.click();
setTimeout(() => {
const confirmBtn = actionDocument.querySelector('button.btn-primary.ms-1');
if (confirmBtn?.textContent.trim() === 'Interview') {
confirmBtn.click();
console.log(`➡️ Interview initiated for device ${index}. Total: ${interviewInitiatedCount + 1}`);
} else {
console.error(`No confirm button for index ${index}`);
}
}, MODAL_DELAY);
}, batchIndex * INTERVIEW_DELAY); // 3s between each interview
batchIndex++;
interviewInitiatedCount++;
});
return batchIndex; // Return count of devices found in this batch
};
/** Scroll and process devices recursively */
const scrollAndProcess = () => {
// Perform the scroll first
const currentScrollHeight = isWindowScroll
? actionDocument.documentElement.scrollHeight
: scrollElement.scrollHeight;
const currentScrollTop = isWindowScroll
? (actionDocument.documentElement.scrollTop || actionDocument.body.scrollTop)
: scrollElement.scrollTop;
const clientHeight = isWindowScroll
? actionDocument.documentElement.clientHeight
: scrollElement.clientHeight;
const isAtBottom = (currentScrollHeight - clientHeight) <= currentScrollTop + 5;
// Scroll to load more items
if (!isAtBottom) {
if (isWindowScroll) {
window.scrollBy(0, SCROLL_STEP);
} else {
scrollElement.scrollTop += SCROLL_STEP;
}
console.log(`📜 Scrolled. Waiting ${SCROLL_SETTLE_DELAY}ms for virtual list to load...`);
}
// Wait for virtual list to settle after scroll
setTimeout(() => {
const initialCount = processedIndexes.size;
const newDevicesCount = processVisibleFailedDevices();
const actualNewCount = processedIndexes.size - initialCount;
if (actualNewCount > 0) {
console.log(`✅ Queued ${actualNewCount} new devices (${newDevicesCount * INTERVIEW_DELAY}ms to process)`);
}
// Calculate wait time: time for all interviews in this batch to complete
const interviewsWaitTime = newDevicesCount > 0
? (newDevicesCount * INTERVIEW_DELAY) + MODAL_DELAY + 500
: 0;
// Check scroll height after processing
const newScrollHeight = isWindowScroll
? actionDocument.documentElement.scrollHeight
: scrollElement.scrollHeight;
// Check if we're done
if (isAtBottom && newScrollHeight <= lastScrollHeight) {
const finalWait = interviewsWaitTime + 1000;
setTimeout(() => {
console.log(`\n🎉 FINISHED! Initiated re-interview for ${interviewInitiatedCount} devices.`);
console.log("Remember to power-cycle mains devices and wake up battery sensors.");
}, finalWait);
return;
}
// Continue scrolling after interviews complete
lastScrollHeight = newScrollHeight;
setTimeout(scrollAndProcess, interviewsWaitTime);
}, SCROLL_SETTLE_DELAY); // Wait 2s for virtual list to render
};
scrollAndProcess();
}
reinterviewFailedDevicesDeepScan();—- TODO: Figure out why the z2m devices didn’t exactly replace the existing zha devices (had to manually update scenes/scripts, and update device areas) |
Beta Was this translation helpful? Give feedback.
and others
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
@Koenkk First of all, love the project! Thank you! I'm so glad to be able to set light gradients on my Signe again, and the software updates are a welcome plus (for no reason other than feeling up to date).
So, I've had a fun night and automated the device migration part with a script:
database.dbentry, e.g.:{ "id": 3, // next highest id "type": "EndDevice", // this will be updated by interview process "ieeeAddr": "0x0017880110409015", // reformatted ieee from zigbee.db "nwkAddr": 19243, // nwk from zigbee.db "manufId": 4107 // endpoint attribute mapped from zigbee.db, mapped via Z2M's cluster.ts }zigbee2mqtt/database.dbzigbee2mqtt/configuration.yaml:This could be automated further for a full ZHA import:
I guess you know all this already, but I've read quite a few forum discussion and watched youtube videos where people crawl through their homes, re-pairing all of their devices manually 🤭
I could write this as a standalone TS script if you're interested?
Beta Was this translation helpful? Give feedback.
All reactions