Hi! DIY Home automation system build.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts
  • paulockenden
    Automated Home Legend
    • Apr 2015
    • 1719

    #16
    Interesting that you consider RF to be “old tech”.

    What's the more modern replacement then?

    Comment

    • paulca
      Automated Home Jr Member
      • Jul 2019
      • 31

      #17
      Originally posted by paulockenden View Post
      Interesting that you consider RF to be “old tech”.

      What's the more modern replacement then?
      Wifi, bluethooth, ethernet. But you could argue the first two are RF. So maybe I should have been clearer and specified 433MHz non-digital RF. Like key fobs.

      Still the same old mine field with switches though. Of course they are all designed to work with their own product range and they don't publish the protocol etc. So I will need to research and find some devices people have successfully paired receivers to.

      Comment

      • paulca
        Automated Home Jr Member
        • Jul 2019
        • 31

        #18
        So just a little update on stuff I did over the weekend.

        This started out as just pure geeky fun but I put temp probes on the heating in and out pipes. Producing this graph:



        The first 4 runs of the heating where normal, either scheduled like this mornings runs or playing around like saturday night, but on the 5th (notably lower) I tested what happens if I point a temperature schedule at the heating itself. In this case I set a 40*C minimum on the heating pipes and sure enough it regulated the heating temperature at 40*C. I'm wondering if I should couple this into the normal schedules or move it's functionality to the heating controller. Yes, I know I can sort of do this with the boiler themostat, but I found it less than accurate and only goes down to 60*C.

        I also graphed the presence and heat status flags, here's the heating one, bonus points if you spot the bug.


        ... and for reference the temperatures:


        Due to the distributed, modular and fairly stateless nature of the design it allows a lot of flexibility. For example I was running the new heating loop schedule on a completely different PC, my main office dev PC at the same time as the existing scheduler was running on the Raspberry PI and they shouldn't trample on each other, well unless you deliberately try and put them into competition with each other.

        I also thought I would share some of the high level code with you. This is what I use to actually set the system up, sparing you the detailed code of "how" it works, for that scroll up and find the GitLab link I shared.

        Defining "the weekend" as a set of time conditions:
        Code:
            all_day_start = datetime.time(hour=6, minute=30, second=0)
            all_day_end = datetime.time(hour=23, minute=00, second=0)
            all_day = TimeCondition(all_day_start, all_day_end)
            weekend = DayOfWeekCondition([6, 7])
            weekend_routine_conditions = [("AND", all_day), ("AND",weekend)]
        The "week day" routine schedules are slightly more complex with morning and evening portions:
        Code:
            weekday = DayOfWeekCondition([1,2,3,4,5])
        
            morning_start = datetime.time(hour=6, minute=30, second=0)
            morning_end = datetime.time(hour=8, minute=00, second=0)
            morning = TimeCondition(morning_start,morning_end)
            weekday_morning_routine_conditions = [("AND", weekday), ("AND", morning)]
        
            evening_start = datetime.time(hour=17, minute=00, second=0)
            evening_end = datetime.time(hour=23, minute=00, second=0)
            evening = TimeCondition(evening_start, evening_end)
        
            weekday_evening_routine_conditions = [("AND", weekday), ("AND", evening)]
        Define the target temps. This blob of data, which is hardcoded, is on the list for wrapping in a network service (Probably within the TemperatureManager) to permit updating it from HID/UI devices like my phone or a wall panel touch screen.

        Code:
            target_temps = {
                "default": {"livingRoom":6, "bedroom":6, "garage":1, "office":6},
                "eco": {"livingRoom": 18, "bedroom": 16, "garage": 2, "office": 18},
                "preferred": {"livingRoom": 20, "bedroom": 18, "garage": 2, "office": 20},
                "max": {"livingRoom": 30, "bedroom": 30, "garage": 30, "office": 30}
            }
            temp_manager = TemperatureManager(target_temps)
        With those definitions and condition sets out of the way I can create schedules. My current go-to schedule is the MultiRoomTempSchedule. You give it a reference to the temp manager and which target to aim for and it will raise a heating demand for which ever target temp is not being hit currently, even if that's multiple rooms at once.

        The 24/7 365 minimum or maintence schedule is simple, it has no time conditions and targets "default":
        Code:
            maintenance_schedule = MultiRoomTempSchedule([], "Maintenance Schedule", temp_manager, "default")
        The "routine" schedules are again multi-room temp schedules targeting "eco" temps.
        Code:
            weekend_routine_schedule = MultiRoomTempSchedule( weekend_routine_conditions, "Weekend Routine", temp_manager, "eco" )
        
            weekday_morning_routine_schedule = MultiRoomTempSchedule( weekday_morning_routine_conditions,
                                                                      "Weekday Morning Routine", temp_manager, "eco" )
        
            weekday_evening_routine_schedule = MultiRoomTempSchedule(weekday_evening_routine_conditions,
                                                                     "Weekday Evening Routine", temp_manager, "eco")
        Finally the real smart bit is the presence schedule, which turns out is just a MultiRoomTempSchedule() at heart at present. I created the new sub class anyway as I hope to extend this to be multi-room presence away, so it can target "eco" in rooms I'm not in and "preferred" in rooms I am in.

        Code:
            presence_condition = PresenceCondition("presenceHome")
        
            presence_schedule = PresenceTempSchedule([("AND",all_day),("AND", presence_condition )], 
                                                                              "Presence Schedule", temp_manager, "preferred")
        "presenceHome" by the way is the data which is published stating if I am in the house or not, based on the phone's Wifi MAC being online.

        Finally, I create a list of these schedules and pass them all to a scheduler:
        Code:
            scheduler = Scheduler([weekend_routine_schedule,
                                   weekday_morning_routine_schedule,
                                   weekday_evening_routine_schedule,
                                   presence_schedule,
                                   maintenance_schedule])
        The main heartbeat of the scheduler is just this time loop:
        Code:
            while True:
                scheduler.update()
                time.sleep(30)
        I can see I have my work cut out for me to provide interfaces to these things. It would not be too hard to run something like REST in the same process scope as the scheduler and modify the various parameters like the target_temps or the times on the routine schedules, the difficult bit is that they then need to be persisted so if the thing power cycles it doesn't just go back to the coded defaults.

        On "power cycling" the system is fully resilient, or maybe it's more that it doesn't care. You can just shutdown, stat up or restart any component and while you obviously lose that functionality while it's down, nothing really gets upset.

        If for example I shut the scheduler off while the heating is on. The heating demand will expire after 5 minutes and the heating will shut off.

        If you shut the heating_hub component off, all the current working data of temps, presence, demands will disappear, but the rest of the system will just shrug and carry on working. The temp sender probes send their data by UDP so they don't even care that anything received the data as they can't do anything about it anyway.

        If I accidentally knock a temp probe plug out of the wall or one of them malfunctions, the temp conditions are smart enough to detect and ignore stale data. Data that is more than 5 minutes old is ignored. So it won't start firing the heating on all day because the living room temp probe died after sending a temp of 18.6*C and never sent another.

        EDIT: What I probably do need is "out of range" detection or "out of normal" detection. For example if a probe fails and starts sending -127*C it should be smart enough to ignore that. Using a "maximum differential" between two indoor temps would help detect "out of normal" readings, but would go in the right direction towards "open window detection".

        Asides "open window detection" the other feature I would like is prediction for what I call "pre-ramp" and "overshoot" on the heating targets. At the moment my 6:30 in the morning is set that way to give the heating time to warm up before I get up at 7am. It would be nice if the scheduler figured that out on it's own. I have ideas about how to do this by running schedules multiple times giving it dates further in the future to find out if any schedules "will" come active soon. If a schedule will be active in 1 hour and the current temp is low by 1 degree I might not preempt it, but if the current temp is 5 degrees below target I might preempt it.
        Last edited by paulca; 1 September 2019, 05:45 PM.

        Comment

        • paulca
          Automated Home Jr Member
          • Jul 2019
          • 31

          #19
          So the issue I'm facing currently is less to do with the control system and more about my house. I only bought it in July.

          The living room seems to be the coldest room in the house. It's also got a very weak radiator. I have just accepted a quote to upgrade my heating from oil fired to gas which will include replacing a few dodgy radiators, including the living room. The downstairs of the house is also fairly open, the kitchen has no doors, so openly joins into the hall and the dinning room and while the living room has doors, they are usually left open.

          However at the moment what happens is the living room drops below 20*C and the heating comes on. The radiators in the kitchen, dinning room and hallway support it's weak radiator and heat it up, the heating goes off, but the living room cools again fairly rapidly. Of course all that heat just moves upstairs making the upstairs too warm even though the upstairs (except bathroom) rads are off. I'm fairly sure this just pulls cold air up through the floor from the dead space into the living room.

          In short it is costing me too much heating just to keep the living room warm, even when I'm not in it.

          Today for example I will be working from home, so I will be in the office upstairs. When I worked from home on Monday the heating was on about 40% of the time just to keep the living room warm.

          Without individual room presence detection (yet) I have decided to manually intervene and lower the target temperature for the living room to 19*C, maybe even 18*C. I will then need to manually intervene again to lift it to 20*C when I finish work and want to watch TV on the sofa. However it should save on oil.

          This is all, I hope temporary. When the new radiator in there I can close it's doors to keep it's heat in which should address the issue. However it does raise a type of requirement for "profiles" again. A working from home (or "in the office") profile in place of multi-room present awareness, which is still only a pondering with bluetooth radars in the temperature probes at the minute. Simply selecting a profile with an expiry would allow temporary altering of target temperatures and stop me having to heat the whole house to satisfy rooms I'm not in.

          I also plan to survive one winter here and then fork out for proper loft and wall insulation. I was well aware of the houses energy rating being dire at an F- with no cavity wall insulation and 1970s roof insulation. It will be interesting to see how I do with it over winter and what difference adding proper insulation makes for the next winter. Part of me is not looking forward to this winter though. I can see me huddling in one room with the doors shut trying to keep one space warm rather than run the heating constantly in Jan/Feb.
          Last edited by paulca; 4 September 2019, 07:52 AM.

          Comment

          • DBMandrake
            Automated Home Legend
            • Sep 2014
            • 2361

            #20
            Originally posted by paulca View Post
            Wifi, bluethooth, ethernet. But you could argue the first two are RF. So maybe I should have been clearer and specified 433MHz non-digital RF. Like key fobs.
            433Mhz key fobs (if you mean car key fobs) are all digital. As are most other things that use 433Mhz like weather stations.

            It's been a very long time since any sort of remote control was analogue...

            Comment

            • paulca
              Automated Home Jr Member
              • Jul 2019
              • 31

              #21
              Originally posted by DBMandrake View Post
              433Mhz key fobs (if you mean car key fobs) are all digital. As are most other things that use 433Mhz like weather stations.

              It's been a very long time since any sort of remote control was analogue...
              Technically no RF is digital, not Wifi, not GSM. Mother nature does not do digital.

              But I take your point that the payload is digitally encoded.

              It is the analogue nature of RF that scares me from an electronics perspective. Dealing with I2C or SPI control over an interface that ends up on Wifi or whatever I'm fine with, but if I had to design a board with an RF chip on it, the mathematics becomes intense very quickly.

              Comment

              • jvallis
                Automated Home Jr Member
                • Feb 2018
                • 29

                #22
                Very interesting. A few things I'd mention:

                1. Your system seems to be operating at quite a high resolution. Boilers, particularly oil burners don't like to be cycled too much - it's best to limit boiler cycles to between 3 and 6 per hour (gas boilers can cycle more). Starting a boiler is a very energy intensive operation and the trick with boilers is to get the condenser working as long as possible once it's heated up. A condenser is best thought of as a turbocharger for a boiler. In this, I would consider using a PID to control the boiler if you don't have the capability to run Opentherm. Alternatively, since you're replacing your boiler and are a geek, I'd strongly suggest you get an Opentherm capable boiler and use Cyril's Opentherm Gateway to monitor and control the boiler.

                2. Given you already have ESP-based sensors, it may be an idea to poll humidity as well and use the humidex to assuage temperatures which is a relatively simple calculation. This is the 'feels like' temperature. Humidity changes when humans are in a room and shaded rooms are different to those with solar radiation (big windows).

                3. Cacti is very old, and has largely been superceded by Influx DB and Grafana, two products that work together very well. This will give you pretty HTML5 graphs that you can zoom in and out etc, and Influx is already a JSON db that's optimised for time-based data.

                4. Please share your work! ESP programs, controller programs etc on github!

                Comment

                • paulca
                  Automated Home Jr Member
                  • Jul 2019
                  • 31

                  #23
                  Hi, thanks for your thoughts.

                  Asides from the experiment of controlling the heating loop temp at 40*C, which I don't think I really need to do, the boiler really only cycles at most 2 or 3 times an hour. Usually it's once once or twice. The current boiler is an old 1970s oil burner, so no condenser. I'd honestly prefer to have independant control of the burner and pump, but I am upgrading it to gas this autumn anyway. Getting a Bosch-Worchester 30kW condensing gas combi-boiler fitted. I will hopefully be able to talk the spark installing the electrics into giving me a plug socket beside the boiler for gadgets as the remote switch on it is "Volt free", ie. a live switch only, no neutral, no power.

                  I'll look into OpenTherm.


                  Humidity. I currently record outdoor humidity, although I have a TODO list to move those outdoor/garage sensors off the RPI and onto an ESP8266 as the PI has become unreliable. It's not just Wifi either, the log is full of fails to read sensors AND no connection to Wifi when it goes funny, so something more fundamental is happening to the hardware, could be spiders, could be cold, could be humidity.

                  I would see the Humindex as more of a thing for cooling, rather than heating, but as next summer I might try automating the air con in the bedroom so it's already cool for heading to bed I keep it as an option.

                  Cacti has released a fairly new version, 1.2.5 which has an all new UI. I do have reservations about it as it's gone "single page webapp" with stateful buttons which I never liked. If I click button X to go to page X, then pressing button X again should refresh page X, but single page web app developers seem oblivious to this concept and that urks me.

                  The question about graphana et. al. Will they read raw RRD files? I'll maybe have to take a look at these, but cacti currently works fine enough. Note I don't use Cacti for data gathering, I manually write RRDs from Python and just point Cacti at the RRDs. RRDs are fairly perfect for timeseries data, but I'm not in that reluctant to try other means, I can always dump the RRD to XML and reimport to something else.

                  On an unrelated point, while I am very happy with the timed, expiring, demand based nature of the heating control, I think it's the right solution for safety, it is however a little cumbersome for control of things like lights. Mostly the polling nature of demands and the single sided state machines (turn it on, just let it time out), don't work well for lights. When you hit a switch you want an instant response from the lights and you probably don't expect the light to go off 15 minutes later unless you press the switch again. My current thoughts are to move to MQTT for lights.

                  My Source code is here: https://gitlab.com/paulcam/home_heating

                  The ESP code in: https://gitlab.com/paulcam/home_heat...master/esp8266 (some legacy)
                  The demand scheduler etc in here: https://gitlab.com/paulcam/home_heat...ster/scheduler

                  Comment

                  • paulca
                    Automated Home Jr Member
                    • Jul 2019
                    • 31

                    #24
                    A small update, purely graphical. I spend half an hour working with Cacti to combine the graphs:

                    Ambient Temperatures
                    Heating Status
                    Presence Home

                    To create this beauty. All the information in one place. Green shading is "Presence" and Blue is "Heating status"


                    You can see an instance of my Raspberry PI in the garage bugging out for a few hours on Thursday morning.

                    As another update, I successfully processed the Heating Pipe graph using the command line to give me a figure of "total hours boiler time", which is a more accurate figure for how much oil/gas I am using given that the boiler is cycling itself to maintain it's loop thermostat. This was done by dumping the RRD for a given time range and when the current loop temp is higher than the previous it adds the time between the current and previous data points to the total run time. If the temperature was the same or lower the boiler is not running.

                    I have yet to make this re-usable. it was just an experiment requiring a bit of shell foo like cat, cut, grep and a 10 line python script to do the detection and counting.
                    Last edited by paulca; 11 September 2019, 10:33 AM.

                    Comment

                    • jvallis
                      Automated Home Jr Member
                      • Feb 2018
                      • 29

                      #25
                      Originally posted by paulca View Post
                      Hi, thanks for your thoughts.

                      Asides from the experiment of controlling the heating loop temp at 40*C, which I don't think I really need to do, the boiler really only cycles at most 2 or 3 times an hour. Usually it's once once or twice. The current boiler is an old 1970s oil burner, so no condenser. I'd honestly prefer to have independant control of the burner and pump, but I am upgrading it to gas this autumn anyway. Getting a Bosch-Worchester 30kW condensing gas combi-boiler fitted. I will hopefully be able to talk the spark installing the electrics into giving me a plug socket beside the boiler for gadgets as the remote switch on it is "Volt free", ie. a live switch only, no neutral, no power.

                      I'll look into OpenTherm.
                      I can understand why you'd want to mess around with a pump and burner on such an old boiler, but modern boilers - like cars - are much more clever and advanced. It's best instead to think of them as a 'black box thing' and use an interface to get the information to monitor and control it, while using it's internal firmware to manage itself, the burner and pump etc.

                      That's why I mentioned Opentherm, a standard developed by Honeywell to do just that. You 'demand' heat by specifying a temperature and the boiler will automatically manage what the burner needs to do to optimise the gas usage to deliver you water at that temperature, and thus depending on the heat loss of your pipes you can generate just the right amount of heat to satisfy the demand at that specific time.

                      Opentherm is a 2-wire serial interface (similar to I2C) that has a set of predefined messages for communication. Cyrils OTGW is on an OrangePi (a Chinese RPi which has Wifi) and a HAT, but it's based on the work done on the PICS-based OTGW (http://otgw.tclcode.com). On that site, you can go to: http://otgw.tclcode.com/matrix.cgi#boilers, and you will see all the message ID's of Opentherm (rollover the number on the left column to see the details of the message). It's best to think of OT as a sliding scale of control rather than just a binary on/off.

                      Worcester Bosch boilers use a proprietary standard called EMS, and you'll need an intermediary device (https://myboiler.com/opentherm/worce...sch-opentherm/) to work with OT. The myboiler site has a few other boilers that support OT.

                      Originally posted by paulca View Post
                      Humidity. I currently record outdoor humidity, although I have a TODO list to move those outdoor/garage sensors off the RPI and onto an ESP8266 as the PI has become unreliable. It's not just Wifi either, the log is full of fails to read sensors AND no connection to Wifi when it goes funny, so something more fundamental is happening to the hardware, could be spiders, could be cold, could be humidity.

                      I would see the Humindex as more of a thing for cooling, rather than heating, but as next summer I might try automating the air con in the bedroom so it's already cool for heading to bed I keep it as an option.
                      It's up to you. Older houses feel colder because they suffer from humidity even in winter. Internal humidity gets changed a lot with heating as it dries it out, and if you get any mildew or condensation, you have a humidity problem usually as a result of over or under ventilation. At the same time you'll get a very crusty nose when waking up in a super dry house. Indoor humidity should always be under 60%, but the range of 30-50% is 'comfortable'.

                      Originally posted by paulca View Post
                      Cacti has released a fairly new version, 1.2.5 which has an all new UI. I do have reservations about it as it's gone "single page webapp" with stateful buttons which I never liked. If I click button X to go to page X, then pressing button X again should refresh page X, but single page web app developers seem oblivious to this concept and that urks me.

                      The question about graphana et. al. Will they read raw RRD files? I'll maybe have to take a look at these, but cacti currently works fine enough. Note I don't use Cacti for data gathering, I manually write RRDs from Python and just point Cacti at the RRDs. RRDs are fairly perfect for timeseries data, but I'm not in that reluctant to try other means, I can always dump the RRD to XML and reimport to something else.
                      RRD is Round Robin Database. It's a database in itself, albeit a very simple one. Influx is another more powerful database, and it's an actual server that communicates with JSON over HTTP. No XML, no RRD no other intermediary format. You can get your ESPs to write directly to Influx by simply POSTing and HTTP request and GETing the result. Furthermore, it's much more powerful than RRD because you can do more complex queries much like SQL (though you again send a special JSON message to do so instead of a SQL string). Just think of it as RRD v2 and saving you the need to translate between JSON, XML and RRD all the time.

                      Grafana is a replacement to Cacti, and can easily be made to read RRD files with an intermediary translator (e.g. https://github.com/doublemarket/grafana-rrd-server). But it works natively with Influx.

                      Originally posted by paulca View Post
                      On an unrelated point, while I am very happy with the timed, expiring, demand based nature of the heating control, I think it's the right solution for safety, it is however a little cumbersome for control of things like lights. Mostly the polling nature of demands and the single sided state machines (turn it on, just let it time out), don't work well for lights. When you hit a switch you want an instant response from the lights and you probably don't expect the light to go off 15 minutes later unless you press the switch again. My current thoughts are to move to MQTT for lights.

                      My Source code is here: https://gitlab.com/paulcam/home_heating

                      The ESP code in: https://gitlab.com/paulcam/home_heat...master/esp8266 (some legacy)
                      The demand scheduler etc in here: https://gitlab.com/paulcam/home_heat...ster/scheduler
                      You might want to also have a look at what Adrian Kennard (the owner of an ISP called AAISP) is also doing with ESP widgets: https://www.revk.uk/

                      The reason I'm encouraging you is that I think ESP is really the way to go, and if we can get to the point that we can have smart ESP sensors and a simple controller to manage them I think this is the next big thing for IoT-based home automation and be completely open source in the process. So many thanks for your engineering so far!

                      Comment

                      • paulca
                        Automated Home Jr Member
                        • Jul 2019
                        • 31

                        #26
                        On OpenTherm. You are right that brute forcing the relay ON/OFF on the power feed to the boiler is certainly not a wise option on a new gas boiler, even though it might appear to work. The option I was considering was placing a relay on it's "Volt less" thermostat lines as if it were using a single wired mechanical thermostat. I'm fairly sure this is still supported.

                        Set-Point temperatures concern me though. This is how things like HoneyWell work. However to implement what I currently have with such a system would cost me hundreds or closer to a thousand pounds in sensors and hubs and .. of course... only work with HoneyWell devices, although I believe there are some hacks into the APIs and integrations with more open hubs.

                        Maybe taking too simplistic a view on set-points I formed the following questions.

                        Which set-point? Living room? Office? Bedroom?
                        What sensor - where does the data come from for it to decide?
                        Different set-points? I don't want all my rooms to be at the same temperature. Nor do I want it turning the boiler down because the living room is 21*C when I'm trying to heat the garage that is -2*C.
                        Foreign control in the system - What if I have more than one way to heat things and the gas boiler is only one of them, the rest beyond it's knowledge or control? Can I tell it to "stay out of it".
                        How does multi-zone control work and what if the radiators valves are not specifically designed for that boiler's proprietary protocols?

                        What if I want to do my own set-point logic, monitor the sensors and tell the boiler to target a "loop temperature" rather than a room temperature? Then control the radiators myself etc.

                        Maybe OpenTherm has all this covered I don't know. If it's just a remote control for the heating hub built into the boiler and I have to have sensors and radiator valves designed for that boiler I'm probably not interested. Not unless OpenTherm provides a way to bridge and or interface outside devices into that system, even then I'd be resistive, although I can always send it faked readings to control it, but that's hacking.

                        On InfluxDB. The thing you seem to point to as a failure of RRD is it's greatest strength and why it is the industry standard for such storage and has been for many decades. It's simple, lightweight, fast, does ONE single job and does it well. It doesn't collect data, it doesn't display data (though you could argue as rrdgraph is part of package that it does, but it's different application and you are not required to use it), it doesn't have a server etc, it just stores data and provides ways to query it. In this sense it follows the Unix convention of small components that do small jobs extremely well that can be built up together to produce much more complex solutions. As an engineer and tinkerer this is the ethos I subscribe to and it is why that even after nearly 50 years Unix-like OSes are still going strong.

                        InfluxDB is more the Windows ethos, or one big application that does n dozen different things, jack of all trades, master of only a few.

                        InfluxDB page one. Run the server. So it requires a network server. Why? I don't need one. Compared to RRD this multiplies the complexity immediately. So it will need to provide something in return that makes it worth it. I can think of a few use-cases that a central server with a query engine could solve for me, so I'm not ruling this out. One such use case is gradient analysis for pre-ramp and over-shoot on heating targets. Querying historic data to work out how quickly the heating can raise the temperature by 2*C and turning the heating on early is an open item on my list.

                        One very serious black mark is that it has "Enterprise" and "cloud" versions. While those "could" just be support and management agreement options to make a bit of money, they often end up with "This feature is only supported in the Enterprise edition" clauses. If not now, when the bean counters get involved and get all Golham on it. Then instead of Enterprise requirements alone it starts to be useful and popular features that move to the licensed model.

                        I will definitely consider it. I need to review the data layer of my system anyway, to open it up to things like HomeAssistant and intergrate with other systems. Moving away from my in memory JSON "blob" will be a step up in complexity, but it might be a good move.

                        On humidity, you might be right. The home survey when bought the place was able to see evidence of mildew on ceilings, even thought they were freshly painted. He suggested "life style choices" around ventilation and heating. With the place currently being sub standard in terms of insulation with only 100mm attic insulation, no cavity wall insulation and no underfloor insulation it will be an interesting winter and I may have to increase the amount of heating I do even while not at home to keep damp at bay.

                        Currently the states are Outside: 83% Living Room: 44% (from the wall clock only as I have on indoor humidity sensor).

                        Comment

                        • paulca
                          Automated Home Jr Member
                          • Jul 2019
                          • 31

                          #27
                          On ESPs. Yes I totally agree they are an amazing platform to do all manor of things.

                          The one downside of them in power consumption. They can consume around an average of 30-40mA of current. This pretty much excludes them from being battery powered. Even a large 3000mAh Lithium Ion would be dead in a day or two.

                          That said there are products such as the SOnOff RF Bridge which adds RF support to the ESP eco-system and allows battery operated switches and sensors that have battery life of months.

                          To that end I did this on Saturday.


                          It's RFSwitch->ESPurna(RFBridge) -> MQTT -> Custom MQTT subscriber -> Raises GroupLightDemand and my normal mechanics take over.

                          I'm not overly thrilled with the delay and for lights I need a more event based structure than the polled "demands".
                          Last edited by paulca; 16 September 2019, 11:54 AM.

                          Comment

                          • paulca
                            Automated Home Jr Member
                            • Jul 2019
                            • 31

                            #28
                            Just one final comment on InfluxDB.

                            It is not round robin. I see some people see this as an advantage. I like the round robin nature, the files never grow. I never need to worry about running out disk space on a a 4Gb SD card and I don't care that older data drops in resolution and eventually dies. I prefer it that way. I don't think I care to have 1 minutes accuracy of data 10 years old.

                            I think my current RRDs are good for 10 years of data and consume exactly and always 4.3Mb of disk space.

                            I wonder how much diskspace InfluxDB will consume in 10 years, if it's even still around then.

                            Comment

                            • paulca
                              Automated Home Jr Member
                              • Jul 2019
                              • 31

                              #29
                              So an update after a long pause.
                              In November I had my heating system changed to natural gas and a combi boiler. I spent the winter with just a 3 point timer.

                              With the Covid situation emerging in March and me working from home I found the time to reimplement the heating controls. This time I put my SOnOff in parallel with the standard day timer.

                              Until July it was just bug fixes and tweaks, in particular balancing my default target temperatures so that the heating would not be coming on when unnecessarily on spring and summer mornings. To do this I lowered the target temp for the living room which is the coldest room in the house when it's not in use.

                              I added an "Override" feature which takes a zone name, a target temp and a number of seconds for it to apply. This overrides all scheduled target temps for that zone. So I can set the living room to 16*C and it won't waste heating while I'm in the office upstairs all day. But when I've finished work or want to go watch a movie I can press a button to set the target temp for the living room to 20*C for 3 hours, knowing it will revert to 16*C automatically I don't need to remember to turn it back down. - Keeping the design ethos of - revert to safe and efficient policy with everything timing out on it's own unless action is taken to persist it -.

                              However late July my SDCard in my Raspberry PI failed. It didn't just start producing errors, it died instantly and completely. "No Medium Found - Please insert disc in drive F:"

                              While my code was all in gitlab, the RRDs with my 3 or 4 years of data were lost. As was all the configuration on Cacti for the graphs.

                              So, I decided I would give the modern alternatives suggested above a go. InfluxDB and Grafana.

                              To be honest, while I do still see some disadvantages of this tech stack the advantages probably out weigh them. Grafana is so much easier to set up graphs for. The default template graph panels are much better and the configuration beyond that much more intuitive than Cacti.

                              So here are some pretty pictures of the new setup:
                              Basic "gauge" dashboard.


                              New Temperature graphs:


                              Solar Panel:


                              The brand new power monitor with a ShellyEM - So new it's graph is less than 24 hours and it already proved useful as it indentified I had an electric supply fault that I was able to report and was fixed before it caused damage across the whole street.

                              Comment

                              • paulca
                                Automated Home Jr Member
                                • Jul 2019
                                • 31

                                #30
                                And finally the visualisation I have been trying to produce for a while and finally got something nice working. The active demands for heating/lighting.


                                You can see around 7:30 when the heating came out of overnight mode into day mode, both the living room and bedroom demanded heating. As I have yet to put thermo-electric valves in yet, the only thing the system can do is demand the actual boiler come on to meet the demand.

                                (The presence graph is noisy because the phone keeps going to sleep every hour or so and dropping off Wifi for a while). EDIT: It (presence) was original made a Demand as demands have an expiry life stamped onto them, where are data points do not. But I'm not exactly sure it needs to be both a datapoint and a demand, so I might remove it from demands.

                                One of the largest changes behind the scenes was moving all the python micro-services that were running on the PI as basic Unix services into Docker using Docker compose. So now it's kind "deploy anywhere". I didn't even have to reflash my devices, I just moved the Raspberry PIs IP address to my Docker host and it all started working again.
                                Last edited by paulca; 28 August 2020, 01:51 PM.

                                Comment

                                Working...
                                X