Analyzing evohome RF protocol - site survey

**zxdavb** · 23rd October 2020, 02:05 AM

Using evohome_rf (https://github.com/zxdavb/evohome_rf), you can eavesdrop RF traffic, and - by sending crafted RQ packets, get deep insights into your system.

This is about performing a site-survery using a HGI80, or evofw3 (https://github.com/ghoti57/evofw3) running on a nanoCUL.

I know of no way to 'discover' the controller (if you can't just get its address from the controller UI), but it will send a sync packet every 3 minutes or so:

Code:

01:09:36.167 || CTL:777777 |            |  I | system_sync      | FF073F   || {'remaining_seconds': 185.5, '_next_sync': '01:12:41'}

Once you've got the address of a controller, you can ask it the address of the heater control (FC) relay & it's configuration:

Code:

01:21:54.873 || HGI:999999 | CTL:777777 | RQ | zone_devices     | 000F     || {'domain_id': 'FC', 'device_class': 'heating_control'}
01:21:54.899 || CTL:777777 | HGI:999999 | RP | zone_devices     | 000F0... || {'domain_id': 'FC', 'device_class': 'heating_control', 'devices': ['13:111111']}

01:21:54.931 || HGI:999999 | CTL:777777 | RQ | tpi_params       | FC       || {'domain_id': 'FC'}
01:21:54.958 || CTL:777777 | HGI:999999 | RP | tpi_params       | FC241... || {'domain_id': 'FC', 'cycle_rate': 9.0, 'minimum_on_time': 5.0, 'minimum_off_time': 0.0, 'proportional_band_width': None}

...here, it's a BDR91A (TPI doesn't apply to OpenTherm Bridges, which have addresses started with 10: ).

What about the configuration of the stored hotwater (FA), if any (from now, I will remove the `RQ` packets for readability):

Code:

01:21:56.321 || CTL:777777 | HGI:999999 | RP | zone_devices     | 000D0... || {'domain_id': 'FA', 'device_class': 'hotwater_sensor', 'devices': ['07:123456']}
01:21:56.379 || CTL:777777 | HGI:999999 | RP | zone_devices     | 000E0... || {'domain_id': 'FA', 'device_class': 'hotwater_valve', 'devices': ['13:222222']}
01:21:56.436 || CTL:777777 | HGI:999999 | RP | zone_devices     | 010E0... || {'domain_id': 'F9', 'device_class': 'heating_valve', 'devices': ['13:333333']}

And the configuration and status of the stored hotwater:

Code:

01:21:56.494661 || CTL:777777 | HGI:999999 | RP | dhw_params       | 00138... || {'setpoint': 50.0, 'overrun': 0, 'differential': 10.0}
01:21:56.599318 || CTL:777777 | HGI:999999 | RP | dhw_mode         | 00000... || {'active': False, 'dhw_mode': 'follow_schedule', 'until': None}
01:21:56.742529 || CTL:777777 | HGI:999999 | RP | dhw_temp         | 000931   || {'temperature': 23.53}

You can also ask the controller how many zones are configured:

Code:

01:13:11.068 || HGI:999999 | CTL:777777 | RQ | system_zones     | 0000     || {'zone_type': 'configured_zones'}
01:13:11.084 || CTL:777777 | HGI:999999 | RP | system_zones     | 00007... || {'zone_mask': [1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0], 'zone_type': 'configured_zones'}

...this says there are 8 zones.

I wonder how many zones of each type there are:

Code:

01:14:58.205 || CTL:777777 | HGI:999999 | RP | system_zones     | 00087... || {'zone_mask': [1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0], 'zone_type': 'radiator_valve'}
01:14:58.253 || CTL:777777 | HGI:999999 | RP | system_zones     | 00090... || {'zone_mask': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'zone_type': 'underfloor_heating'}
01:14:58.301 || CTL:777777 | HGI:999999 | RP | system_zones     | 000A0... || {'zone_mask': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'zone_type': 'zone_valve'}
01:14:58.348 || CTL:777777 | HGI:999999 | RP | system_zones     | 000B0... || {'zone_mask': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'zone_type': 'mixing_valve'}
01:14:58.396 || CTL:777777 | HGI:999999 | RP | system_zones     | 00110... || {'zone_mask': [0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0], 'zone_type': 'electric_heat'}

... the last packet tells me that the 8th zone is an electric heat type.

You can get the sensor, the actuators, and the configuration & status of a zone:

Code:

01:33:45.242896 || CTL:777777 | HGI:999999 | RP | zone_devices     | 01000... || {'zone_idx': '01', 'device_class': 'zone_actuators', 'devices': ['04:123456', '04:123457']}
01:33:45.301588 || CTL:777777 | HGI:999999 | RP | zone_devices     | 01040... || {'zone_idx': '01', 'device_class': 'sensor', 'devices': ['34:123456']}
01:33:45.370201 || CTL:777777 | HGI:999999 | RP | zone_name        | 01004... || {'zone_idx': '01', 'name': 'Lounge'}
01:33:45.428955 || CTL:777777 | HGI:999999 | RP | zone_params      | 01100... || {'zone_idx': '01', 'min_temp': 5.0, 'max_temp': 35.0, 'local_override': True, 'openwindow_function': True, 'multiroom_mode': False}
01:33:45.486836 || CTL:777777 | HGI:999999 | RP | zone_mode        | 0105D... || {'zone_idx': '01', 'mode': 'follow_schedule', 'setpoint': 15.0}
01:33:46.776924 || CTL:777777 | HGI:999999 | RP | temperature      | 01086A   || {'zone_idx': '01', 'temperature': 21.54}

You can puzzle UFH controllers too, for example:

Code:

13:56:13.072 || UFC:333333 | HGI:999999 | RP | device_info      | 00000... || {'description': 'HCE80 V3.10 061117', 'firmware': None, 'manufactured': '2017-11-06'}
13:56:14.336 || UFC:333333 | HGI:999999 | RP | zone_devices     | 00090... || {'ufh_idx': '00', 'zone_id': '01', 'device_class': 'ufh_actuators', 'devices': ['01:333333']}
13:56:14.558 || UFC:333333 | HGI:999999 | RP | zone_devices     | 01090... || {'ufh_idx': '01', 'zone_id': '05', 'device_class': 'ufh_actuators', 'devices': ['01:333333']}
13:56:14.692 || UFC:333333 | HGI:999999 | RP | zone_devices     | 01090... || {'ufh_idx': '02', 'zone_id': '03', 'device_class': 'ufh_actuators', 'devices': ['01:333333']}
13:56:18.756 || UFC:333333 | HGI:999999 | RP | zone_devices     | 04097... || {'ufh_idx': '03', 'zone_id': None, 'device_class': 'ufh_actuators', 'devices': []}
...

... above, you get the mappings between an UFH curcuit (loop) and the corresponding evohome zone.

This is useful, as the UFH controller will send heat demand for each UFH circuit.

Code:

19:33:32.089 || UFC:333333 |            |  I | heat_demand      | FC8C     || {'domain_id': 'FC', 'heat_demand': 0.7}
19:35:54.717 || UFC:333333 |            |  I | heat_demand      | 008C0... || [{'ufh_idx': '00', 'heat_demand': 0.7}, {'ufh_idx': '01', 'heat_demand': 0.0}, {'ufh_idx': '02', 'heat_demand': 0.64}]

... it seems the overall sent demand from the UFH controller to the evohome controller is the highest of all demands - but the controller can use the second packet if it wishes..

You can pull the controller's fault logs, and zone schedules too.

**philchillbill** · 3rd November 2020, 06:13 PM

Originally Posted by zxdavb

You can pull the controller's fault logs, and zone schedules too.

I got this set up on a nanoCUL so that my HGI-80 can stay dedicated to Domoticz - nice toy

How exactly do you 'pull' the zone schedule with this?

**zxdavb** · 4th November 2020, 10:49 PM

The below is in the bleeding_edge branch of github.com/zxdavb/evohome_rf.

I have been focused upon the client library, not so much on the client - I am hoping others will take that on... So client.py isn't elegant, but:

Code:

python client.py execute /dev/ttyUSB1 --device-id 01:145038 --get-schedule 01

... makes this happen in evohome_rf (whitespace added for readability):

Code:

Starting evohome_rf...

2020-11-04T22:43:22.901809 || HGI:013393 | CTL:145038 | RQ | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 1, 'frag_total': 0, 'frag_length': 0}
2020-11-04T22:43:22.972912 || CTL:145038 | HGI:013393 | RP | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 1, 'frag_total': 5, 'frag_length': 41, 'fragment': '68816DCDCD0902311040E1F9CB043D5881055883D558C09E3C598757610F624116601196601617DD2C'}

2020-11-04T22:43:23.006839 || HGI:013393 | CTL:145038 | RQ | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 2, 'frag_total': 5, 'frag_length': 0}
2020-11-04T22:43:23.051819 || CTL:145038 | HGI:013393 | RP | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 2, 'frag_total': 5, 'frag_length': 41, 'fragment': '2F10088FCC7C22222ADF7369F755DE5B59B4477B1CEB69B36C7B1319EAA1FB777791E7AA5D636AFDEC'}

2020-11-04T22:43:23.087847 || HGI:013393 | CTL:145038 | RQ | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 3, 'frag_total': 5, 'frag_length': 0}
2020-11-04T22:43:23.132812 || CTL:145038 | HGI:013393 | RP | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 3, 'frag_total': 5, 'frag_length': 41, 'fragment': 'AD889CF36F28B80AAE82ABE02AB80AAE816BE01AB806AE816BE03AB80EAE83EBE03AB80E6E801BE006'}

2020-11-04T22:43:23.167809 || HGI:013393 | CTL:145038 | RQ | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 4, 'frag_total': 5, 'frag_length': 0}
2020-11-04T22:43:23.213835 || CTL:145038 | HGI:013393 | RP | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 4, 'frag_total': 5, 'frag_length': 41, 'fragment': 'B8016E801BE01670CBEC8EB56F933B2EF6959FDBB7B55BC04D7013DC0437C14D70737677CDFD003DAB'}

2020-11-04T22:43:23.247833 || HGI:013393 | CTL:145038 | RQ | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 5, 'frag_total': 5, 'frag_length': 0}
2020-11-04T22:43:23.265798 || CTL:145038 | HGI:013393 | RP | zone_schedule    | 01200... || {'zone_idx': '01', 'frag_index': 5, 'frag_total': 5, 'frag_length': 2, 'fragment': '494C'}

Finished evohome_rf.

... and the client can then extract this:

Code:

[{'day_of_week': 0, 'switchpoints': [{'time_of_day': '02:00', 'heat_setpoint': 15.0}, {'time_of_day': '07:00', 'heat_setpoint': 18.5}, {'time_of_day': '09:00', 'heat_setpoint': 19.0}, {'time_of_day': '15:30', 'heat_setpoint': 20.0}, {'time_of_day': '19:30', 'heat_setpoint': 20.0}, {'time_of_day': '23:55', 'heat_setpoint': 16.5}]}, {'day_of_week': 1, 'switchpoints': [{'time_of_day': '02:00', 'heat_setpoint': 15.0}, {'time_of_day': '07:00', 'heat_setpoint': 18.5}, {'time_of_day': '09:00', 'heat_setpoint': 19.0}, {'time_of_day': '15:30', 'heat_setpoint': 20.0}, {'time_of_day': '19:30', 'heat_setpoint': 20.0}, {'time_of_day': '23:55', 'heat_setpoint': 16.5}]}, {'day_of_week': 2, 'switchpoints': [{'time_of_day': '02:00', 'heat_setpoint': 15.0}, {'time_of_day': '07:00', 'heat_setpoint': 18.5}, {'time_of_day': '09:00', 'heat_setpoint': 19.0}, {'time_of_day': '15:30', 'heat_setpoint': 20.0}, {'time_of_day': '19:30', 'heat_setpoint': 20.0}, {'time_of_day': '23:55', 'heat_setpoint': 16.5}]}, {'day_of_week': 3, 'switchpoints': [{'time_of_day': '02:00', 'heat_setpoint': 15.0}, {'time_of_day': '07:00', 'heat_setpoint': 18.5}, {'time_of_day': '09:00', 'heat_setpoint': 19.0}, {'time_of_day': '15:30', 'heat_setpoint': 20.0}, {'time_of_day': '19:30', 'heat_setpoint': 20.0}, {'time_of_day': '23:55', 'heat_setpoint': 16.5}]}, {'day_of_week': 4, 'switchpoints': [{'time_of_day': '02:00', 'heat_setpoint': 15.0}, {'time_of_day': '07:00', 'heat_setpoint': 18.5}, {'time_of_day': '09:00', 'heat_setpoint': 19.0}, {'time_of_day': '15:30', 'heat_setpoint': 20.0}, {'time_of_day': '19:30', 'heat_setpoint': 20.0}, {'time_of_day': '23:55', 'heat_setpoint': 16.5}]}, {'day_of_week': 5, 'switchpoints': [{'time_of_day': '02:00', 'heat_setpoint': 15.0}, {'time_of_day': '07:00', 'heat_setpoint': 19.5}, {'time_of_day': '09:00', 'heat_setpoint': 19.5}, {'time_of_day': '15:30', 'heat_setpoint': 20.0}, {'time_of_day': '19:30', 'heat_setpoint': 20.0}, {'time_of_day': '23:55', 'heat_setpoint': 16.5}]}, {'day_of_week': 6, 'switchpoints': [{'time_of_day': '02:00', 'heat_setpoint': 15.0}, {'time_of_day': '07:00', 'heat_setpoint': 19.5}, {'time_of_day': '09:00', 'heat_setpoint': 19.5}, {'time_of_day': '15:30', 'heat_setpoint': 20.0}, {'time_of_day': '19:30', 'heat_setpoint': 20.0}, {'time_of_day': '23:55', 'heat_setpoint': 15.5}]}]

**zxdavb** · 4th November 2020, 10:52 PM

And also:

Code:

python client.py execute /dev/ttyUSB1 --device-id 01:145038 --get-faults

... makes this happen in evohome_rf (whitespace added for readability):

Code:

Starting evohome_rf...

2020-11-04T22:50:09.743843 || HGI:013393 | CTL:145038 | RQ | system_fault     | 000000   || {'log_idx': '00'}
2020-11-04T22:50:09.770883 || CTL:145038 | HGI:013393 | RP | system_fault     | 00400... || {'log_idx': '00', 'timestamp': '20-10-29T21:19:48', 'fault_state': 'restore', 'fault_type': 'comms_fault', 'device_class': 'dhw_sensor', 'domain_id': 'FA', 'device_id': '07:045960'}

2020-11-04T22:50:09.802874 || HGI:013393 | CTL:145038 | RQ | system_fault     | 000001   || {'log_idx': '01'}
2020-11-04T22:50:09.830894 || CTL:145038 | HGI:013393 | RP | system_fault     | 00000... || {'log_idx': '01', 'timestamp': '20-10-29T18:29:50', 'fault_state': 'fault', 'fault_type': 'comms_fault', 'device_class': 'dhw_sensor', 'domain_id': 'FA', 'device_id': '07:045960'}

2020-11-04T22:50:09.863292 || HGI:013393 | CTL:145038 | RQ | system_fault     | 000002   || {'log_idx': '02'}
2020-11-04T22:50:09.891939 || CTL:145038 | HGI:013393 | RP | system_fault     | 00400... || {'log_idx': '02', 'timestamp': '20-10-15T19:06:42', 'fault_state': 'restore', 'fault_type': 'comms_fault', 'device_class': 'dhw_sensor', 'domain_id': 'FA', 'device_id': '07:045960'}

2020-11-04T22:50:09.920877 || HGI:013393 | CTL:145038 | RQ | system_fault     | 000003   || {'log_idx': '03'}
2020-11-04T22:50:09.948846 || CTL:145038 | HGI:013393 | RP | system_fault     | 00000... || {'log_idx': '03', 'timestamp': '20-10-15T18:16:07', 'fault_state': 'fault', 'fault_type': 'comms_fault', 'device_class': 'dhw_sensor', 'domain_id': 'FA', 'device_id': '07:045960'}

2020-11-04T22:50:09.979886 || HGI:013393 | CTL:145038 | RQ | system_fault     | 000004   || {'log_idx': '04'}
2020-11-04T22:50:10.007849 || CTL:145038 | HGI:013393 | RP | system_fault     | 00000... || {}

Finished evohome_rf.

... and the client can then extract this:

Code:

{'timestamp': '20-10-29T21:19:48', 'fault_state': 'restore', 'fault_type': 'comms_fault', 'device_class': 'dhw_sensor', 'domain_id': 'FA', 'device_id': '07:045960'}
{'timestamp': '20-10-29T18:29:50', 'fault_state': 'fault', 'fault_type': 'comms_fault', 'device_class': 'dhw_sensor', 'domain_id': 'FA', 'device_id': '07:045960'}
{'timestamp': '20-10-15T19:06:42', 'fault_state': 'restore', 'fault_type': 'comms_fault', 'device_class': 'dhw_sensor', 'domain_id': 'FA', 'device_id': '07:045960'}
{'timestamp': '20-10-15T18:16:07', 'fault_state': 'fault', 'fault_type': 'comms_fault', 'device_class': 'dhw_sensor', 'domain_id': 'FA', 'device_id': '07:045960'}

**zxdavb** · 4th November 2020, 10:56 PM

If anyone has a HM80, I would love to see the packet log from:

Code:

python client.py execute /dev/ttyUSB1 --device-id 01:145038 -o packet.log

**philchillbill** · 5th November 2020, 08:51 AM

Originally Posted by zxdavb;

Code:

python client.py execute /dev/ttyUSB1 --device-id 01:145038 --get-schedule 01

Wow, works really nicely - well done and thanks! I'm guessing that your use of the async pyserial library is making a big difference here. I had very mixed results using DanD's schedule backup/resume scripts which do not use async (as far as I can tell).

I do notice a message that "There is only limited support for Windows' when I run this. Just curious why the platform would make a difference - isn't Python just Python?

**zxdavb** · 5th November 2020, 11:53 AM

Dan did an amazing job decoding this packet type (I'm pretty good at decoding packets, but I wouldn't have been able to do it), but his code is more proof-of-concept, I guess. In addition, my library does not currently write schedules (this is planned).

The speed/reliability has nothing to do with async (although it was easier to implement with pyserial-asyncio) - I did a lot of work adding a QoS wrapper around the serial interface: event driven now (no sleeps), with timeouts, retransmits, etc.

This QoS makes everything else so much easier, including max transmit rates without exceeding the duty cycle limit of the HGI80 (evofw3 doesn't have one).

re Windows/Linux: No, some of the deeper stuff is different between Linux & Windows - this may break at anytime:
- reduced granularity of timestamps (so I call Windows APIs directly, but even then...)
- no SIGUSR1/SIGUSR2 (which have been assigned useful functions)
- limited support by pyserial-asyncio (it says it doesn't work, but it does, maybe because of: )
- I use: asyncio.set_event_loop_policy(asyncio.WindowsSelec torEventLoopPolicy())

**philchillbill** · 5th November 2020, 05:59 PM

Many, many thanks are due to the both of you for getting this all working so well.

I'm going to have a look at your QoS stuff and see if I can apply it in some way to Dan's code. I want schedule read/write to be rock solid as I'm writing an Alexa Skill that will allow schedule editing by voice and I do not want to have to rely on the TCC cloud for that, given that Honeywell can pull the API token we all use at any time. I already use Alexa with my own Skill as the primary interface to my Evohome setup, together with Google Calendar for scheduling one-off things that do not repeat every week similarly. Right now, I can do timed overrides like "Alexa, set the office to 22 deg for 20 mins" which is great. But when I have schedule-edit capabilities, that could become "Alexa, make the office 22 deg from 16:00 for 30 mins" which is a lot more powerful. Also, things like e.g. delay or pull-in bedtime by 45 mins if staying up later than usual to finish a movie will be a lot easier to do via on-the-fly schedule edits.

My Domoticz-serving HGI80 is connected to an Intel NUC running Ubuntu, so I can move the Alexa-nanoCUL to that platform from my Win 10 machine if it will work better. I just find developing on a Win box easier.

**zxdavb** · 5th November 2020, 08:00 PM

You're welcome to have a go at it, but: OMG - it's not straight-forward piece of code.

evohome_rf is designed to interface between consumers like your skill and a HGI80/evofw3 device - can you use python libraries in an Alexia skill?

**philchillbill** · 5th November 2020, 09:03 PM

Originally Posted by zxdavb

You're welcome to have a go at it, but: OMG - it's not straight-forward piece of code.

evohome_rf is designed to interface between consumers like your skill and a HGI80/evofw3 device - can you use python libraries in an Alexia skill?

Skills can indeed be written in Python and can include libraries zipped into the upload package. However, the skill (running as an elastic linux lambda function in the cloud) needs to talk to a https endpoint with a domain name in it, so the way I will make it work is with a helper program that runs in the local network of the user and that would have the (modded) evohome_rf functionality in it. I already use Flask to give me a restful interface to DanD's P.O.C. code and that is surprisingly easy to do. It allows http GET and PUT via URL &read or &write parameters which map to command line switches in the python script. ngrok or Apache/nginx then give that Flask instance an external https URL that s Skill can talk to.

Using flask, you could add a restful API to evohome_rf with very little extra coding, allowing it to be run as a web service on a linux box. But that might not be your vision for the code. Have you ever played with Flask?

Thread: Analyzing evohome RF protocol - site survey

Thread Tools

Display

Analyzing evohome RF protocol - site survey

Posting Permissions