Results 1 to 10 of 33

Thread: Hi! DIY Home automation system build.

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Automated Home Jr Member
    Join Date
    Jul 2019

    Default Hi! DIY Home automation system build.

    Hi guys,

    I'm Paul, 44, software engineer from Northern Ireland. I have been slowly hobbying up a smart home system for a while now, I thought I would share some of it with you guys and share some of the ideas for where it goes next.

    So far I have a data network in the home. It's very simple, as complex as it currently needs to be. Devices send JSON blobs via UDP to a hub. The hub merges the JSON into a single lump of data and makes it available to any other device that is interested. Presently the whole dataset is sent if you as much as connect on it's TCP reading port, but this could become a REST API.

    So for example I have a few Raspberry Pis and a few ESP8266's with temp sensors shoved in discrete places. Every 5 seconds they wake, read the temp and send a JSON string to the hub. Something like:
    {"type": "float", "units": "'C", "key": "livingRoom", "value": "21.81",  "shortName": "LR", "name": "Living Room"}
    the hub then adds this to a larger tree of data and adds a timestamp to the json object to allow stale data detection.

    I also have a monitor on my solar panel controller, which uses an RS485 to serial converter and an ESP8266 to transmit it's parameters every 5 seconds to the hub.

    The only client I have currently is a small RaspberryPi enclosure with a 16x2 screen which just displays all of the data 2 at a time on the screen in a loop.

    Additional to collecting and sending data, the hub also, every minute, dumps a snapshot of the data into RRA archives. If new data arrives it will automatically create a new RRA for it. I have a web application called "Cacti" which provides management for graphing this RRAs on demand, allowing me to view graphs of data over the web.

    RRAs are round robin archives. They are fixed size, multi layer, sequence database. Mine store 1 minute resolution for 2 years, 5 minute resolution for 10 years and 1 hour resolution for (I can't remember... a long time). They never get larger, never fill you disk space.

    I am currently moving on to the first real automation. The heating controller.

    I will follow up with a post of the first version approach to heating scheduling and then the future ideas for making it more of a multi-zone, presence aware, demand based system for when I upgrade my heating a bit.

    Here are a few images:
    Solar panel, RS485 to Wifi monitor box:

    Raspberry PI Monitor/Display:

    Graph output from RRAs

  2. #2
    Automated Home Jr Member
    Join Date
    Jul 2019


    So as i said the first thing I intend to control is the heating. Presently I have oil fired heating where the boiler is switched directly by the heating controller relay. The wiring diagram for an external thermostat is, regrettably, just putting another relay in series with the heating power feed. So this means switching current, but not that much. The start up surge current, as best as I can gather is much less than the 10A capacity of a SonOff or an opto-isolated relay. I prefer the SonOff, though I have never used one yet, at least it doesn't involve frankenstiening up an ESP8266 and relay board. The SonOff is just simple, contained and fairly safe.

    So scheduling the heating. Version 1 will just be a true/false output for heating.

    So far I have a basic Python application running on the hub, however due to the distributed nature of the system I already have, the heating scheduler does not need to be on the hub and will actually connect to get it's data, even if that is "localhost" just.

    The general theory/design of version 1 is a Scheduler which is a collection of Schedules. Schedules are in order of priority. The first one to return "Active" sets the heating to ON, if none return active the heating is OFF. The heating requirement will be published back into the JSON data array. A further script/program/MCU will periodicly poll this data looking for something like {"heating": {"state": "True"}} and (assuming the SonOff) send an event to turn the relay on.

    A Schedule is a collection of conditions and their modifiers. I have already written conditions for "Time of day", "Day of week", "Temperature minimum". So, for example, my weekday morning schedule might be:

    TimeOfDay 06:30 - 07:45
    AND DayOfWeek 1, 2, 3, 4, 5
    AND Temperature livingRoom 20

    A few utilitarian schedules will exist with much higher priority, for example:
    Temperature livingRoom 10
    Temperature bedroom 10
    Temperature garage 2

    Another, slightly more complex, schedule will need the last time the heating was run placed in the dataset and based on this value, run the heating for 15 minutes if the heating has not been run in 23:44 minutes. Call it the "exercise schedule".

    There are obviously a few over simplifications and I have a few implementation details proving somewhat tricky, but it should be functional. This is mostly coded and I'm just going to order the SonOff to test it out, with a table lamp at first, then with the heating itself.

    There are also some fail safe items to provide here. For example the timestamp on the "Heating On" data will allow the heating controller to detect the "command" stale is stale and the scheduler should have been refreshing it every minute and will ignore it and turn the heating off. With the SonOff, I can flash the firmware to do similar, to make a single "ON" command only be valid for 1 hour before switching off.

    I have to get to work, so I'll post the Version 2 design, which is were it get's interesting.

  3. #3
    Automated Home Jr Member
    Join Date
    Jul 2019


    So before I outline my design for the heating control scheduling, a small aside.

    I got the SOnOff basic switches in the post. They didn't work at all with the supported software. This was only slightly irritating as I fully intended to flash them with my own software, but I would liked to have seen them function as intended first.

    So the current R2 models use an ESP8285, not an ESP8266 which renders most of the custom firmware options a little more than drop in replacements. The software itself will work, but there are differences in the flashing process etc.

    Anyway I took the default webserver code that comes with the stock ESP8266 and implemented a basic REST api on to the switch. I have three end points:
    http://the.switch/on - turns in on
    http://the.switch/off - turns in off
    http://the.switch/status - tells you if it's on or off

    For the heating switch this will be extended to have:

    1. The button single press will turn the heating on or off, of course the scheduler might just turn it off or back on again. This is primarily for if the hub is down or offline for whatever reason and I need heat.
    2. A long press on the button will override the automation for 1 hour and run the heating regardless of what the hub asks for.
    3. Anti-short cycle. Even though this will be implmented by the gas boiler, I choose to add it anyway. So if the heating has been turned off, it will refuse to turn back on within 1 minute. This prevents broken software from rapidly toggling the heating on and off.
    4. "ON" request timeout after 1 hour. So if the hub crashes when it was about to turn the heating off and leaves the heating running it will timeout after 1 hour unless the /ON command is refreshed periodically.

    These devices are almost all based on ESP8285 and a small amount of research, a 5 USB Serial/TTL converter and the Arduino IDE and you can put your own custom software on them.

  4. #4
    Automated Home Jr Member
    Join Date
    Jul 2019


    Some rather pretty graphs from my system:

    Ambient temperatures. Note the huge spike in the garage temp when the clothes drier is running!

    This should make detecting day and night very easy, solar panel voltages!

  5. #5
    Automated Home Jr Member
    Join Date
    Jul 2019


    So heating control. Now bear in mind I am kind of deliberately doing this "clean room" style. That means I am not really looking at what others are doing or how they do it. I am trying to design this from scratch based on what my view of heating control is.

    Version 1, which is more-or-less coded and needs maybe an evening to finish is basically a more advanced 7 day timer with temperature awareness. However the schedules themselves would be much more complex as I do not need to provide a UI for them, I can simply log into the Raspberry PI hub and edit a few lines of code to change them. Though there is more on that later.

    I would say the code is "simple", but that's subjective. It's aiming to provide simple building blocks from which complexity can be constructed.

    A Scheduler contains a set of Schedules. It basically moves through each Schedule in turn, hands it the current data from the network and asks it if it's active or not. For future proofing the whole set of active schedules is returned in the order they were queried. Order is important as it provides priority. However currently the first active schedule is enough and "wins" control. (Again more on this later).

    A Schedule then is, in it's basic form, is a set of Conditions which the Schedule "evaluates" to determine if it should be active or not. Conditions come in a variety of forms; TimeOfDay, DayOfWeek, DayOrNight, MinimumTemperature, TemperatureBand, etc. etc. Even, though not obvious, HeatingOn condition. Conditions are attached to an operator, such as AND and OR. So when placed in an ordered list these give an expression.

    So an example. Morning, week day schedule:
    OR TimeOfDayCondition 6am - 8am
    AND DayOfWeekCondition Mon-Fri
    OR MinimumTemperature LivingRoom 18*C
    OR MinimumTemperature BedRoom 18*C

    The logic is such that a "False" immediately stops the evaluation. It's a bit brittle this and I'm not happy with it. It takes quite a bit of thought to work out what that series will or won't do for a set of circumstances. But it's version 1.

    If the full expression returns True the Schedule considers itself active and returns "Active"

    If you are still reading... Returning active in version 1 will simply re-publish a new JSON blob, or update the existing one with "HeatingDemand": "True" and also "ActiveSchedule":"Morning Weekday"

    So single zone, single active schedule, single on/off. For now.

    Another script or completely separate device can poll that demand status and send/activate the correct device to turn the heating off.

    There is logic in my madness here. It's keeping things modular. In fact there are the following "modules":

    Sensors and devices.
    The data hub / logging. The heart of the thing.
    The scheduler. As discussed here.
    The heating (in my case) controller.
    The SOnOff device doing the switching.

    The layered separation means that, for example, I can present an HID (interface) to the heating controller allowing me to completely override the heating and take "manual" control. It also allows a layer in which to evaluate "acceptable use" of the device being controlled. Short-cycling, on to long, cycling, minimum time on, etc.

    I can also intervene at the SOnOff switch itself of course.

    The "More on that later"'s will have to wait to the next post, but are mostly about Version 2 which I try to address various issues with the above approach.

    One I'm actually struggling with is negative demands. A real world example. The set up is doing it's job on a Friday evening, it's 22*C, but I'm going out and staying at a friends, so I want a way to say, "Heating off.", or "I'm out" mode. I can, intervene at the "Heating controller", but I want something smarter than just switching it off.

    I think it might be better to play with the target temperatures overriding the active schedule itself.

  6. #6
    Automated Home Jr Member
    Join Date
    Jul 2019


    A little off-topic from heating control, but a cool feature that popped up.

    I have 4 lights in my living room. All are plugged into wall sockets. On an evening I typically have to go around the room and switch each one on at the wall. When it's bedtime I have to go and switch them all off again.

    A light bulb moment (excuse the pun) was when I realised I could have any one of those switches (if a smart switch) raise a demand for it's peers to be on too.

    An afternoon coding and I have created what I call "Grouped Lights".

    My custom firmware on a SOnOff switch allows you to:
    1. Switch it on/off by pressing the button.
    2. Switch is on/off by REST http request
    3. Have it publish a demand for it's group to the network by long pressing the button.

    The demands are published as JSON and a "GroupLightController" Python script polls these and when it sees a change in demand for the group of lights it commands they to swtich on/off via REST.

    Also, here is the code for the whole system as it stands now.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts