System Watchdog

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts
  • Paul_B
    Automated Home Legend
    • Jul 2006
    • 608

    System Watchdog

    My Cortex server hung last night and although I had a previous watchdog setup to reset the nodes and put them in reflex mode this never seems to work.

    Now that I have a DFP module it easily exposes a timer module and I have followed the help instructions to set this up as a watchdog. The watchdog is working and the nodes are getting reset but they don't seem to want to go into Reflex mode.

    I can see the timer event and the node reset occur in Cortex through the communication window with the network paused (as described in the help file). However, a reflex action on a switch is to toggle the associated light and another reflex is to turn on a light when the door opens. Neither action occurred. Anyone any ideas what I maybe doing wrong?

    I am sure my reflex programming is working because in the past if I remove the network power and then turn it on again (with Cortex shutdown) the modules operate as expected in Reflex mode.

    Paul
  • Karam
    Automated Home Legend
    • Mar 2005
    • 863

    #2
    Sorry to hear you've been having difficulties with this. Please feel free to contact us directly and we can help you analyse the detail of the set up you have to get it working correctly.

    In general though a technical note on the principles:

    Reflex programming sets up some behaviours in individual modules through programming which is stored in EEprom (ie. stays resident on the module even after power reset). In the past this programming had to be done manually using the Cortex Reflex programming utility. Manually in that the user had to for example dictate which switch was to operate which light when creating the Reflex program. More recently we have introduced an auto Reflex creation tool at present primarily for lighting and simple heating functions. The tool looks at the existing database and from this surmises which switches are associated with which lights and then automatically creates the necessary Reflexes to allow simple manual operation of the lights. The tool also has provision for automatically setting up a watchdog.

    The action of Reflexes is typically gated by a few parameters whose value can also be programmed into EEprom (would get done automatically by Auto Reflex tool).

    When a module is reset it loads in its working parameters (including the Reflex gating AND initial output states) into working memory and its behaviour is thereafter governed by the settings in working memory. When Cortex runs a network it disables the Reflex functions by changing the settings in working memory. However there are some exceptions. One is: If a watchdog timer Reflex is set up using a module timer and Cortex knows about this then it will not disable the timeout Refllex and furthermore will know to setup and refresh the watchdog timer at regular intervals. If Cortex is unable to refresh the timer because the PC has crashed or power lost to it or such, then the watchdog will eventually time out and trigger whatever Reflex it has been programmed with. For a system without an IPS (intelligent power supply) typically this is a single broadcast packet to reset all modules. When the modules reset they load in their EEprom parameters and until Cortex resumes operations they will now operate in Reflex mode. In a system which uses an IPS it is the default to use the timer in the IPS as the watchdog. On timeout the IPS watchdog then performs a full spur reset i.e physically removes and re-instates power to the modules.

    If the switch over to Reflex isn't working properly it could be that the watchdog timer Reflex is not set up correctly or that the module programming has not been done or is missing some aspect such as ensuring the gating parameters are enabled. Since you think the watchdog bit is working correctly and the modules in question ARE being reset then the next thing to suspect is the Reflex programming on the module. If you have Cortex in a non network running state and manually reset the module attached to the switch in questin then after the module powers up perform a node profile analysis then it will be possible to see what working parameters the module has loaded in - though this will probably have to be sent to us for interpretation.

    I should make the point that this procedure for moving from Cortex operation to Reflex operation is not the only way to do things and could infact be viewed as crude. For example a watchdog Reflex could, rather than resetting all modules, send out packets to enable the working memory Reflex gating in the various modules. This would make a transition to Reflex mode smoother in the sense that any output states do not have to go through a reset. There are other possibilities too but that's another story ..

    Finally we would always advise that installers should check Reflex operation when they deem it relevant eg. just after commissioning or after adding/removing modules and any associated or independent changes to Reflex programming. The test can be as Paul suggested by pausing a running network, or if you want to be a bit more brutal unplug the PCA or shutdown the PC. It is also a good idea to check how your PC reacts to a power cut. The problem with Reflex back up is that in practice it is supposed to be and indeed found to be something that is needed so very infrequently that people forget the importance of checking it after changes are made.

    Comment

    • Paul_B
      Automated Home Legend
      • Jul 2006
      • 608

      #3
      Karam,

      Thanks for the detailed explanation.

      The issue looks to be one that highlighted
      ...just after commissioning or after adding/removing modules and any associated or independent changes to Reflex programming
      I did a few further tests, firstly killing the Cortex process with network running (I backed up the database first in case of corruption) after the timeout period reflex action did not occur. Secondly, I removed the power from my network and after a 15 seconds re-added. In the past this would have got modules into Reflex however this time it didn't.

      I've rechecked some of the reflex actions against modules and noticed that they have been cleared. So I am now attempting to reprogramme.

      One question from my discoveries is what is the difference between the following two reset broadcasts?

      The help section of Cortex states that the packet associated with the watchdog should be:

      FF910010000004000200

      However, I noticed the Auto-Reflex creates a slighlty different packet:

      FF91FFFF000004000200

      I'm starting a separate thread on some specific Reflex programming issues

      Paul

      Comment

      • Karam
        Automated Home Legend
        • Mar 2005
        • 863

        #4
        Originally posted by Paul_B View Post
        The help section of Cortex states that the packet associated with the watchdog should be:

        FF910010000004000200

        However, I noticed the Auto-Reflex creates a slighlty different packet:

        FF91FFFF000004000200
        In the first packet 0010 is the explicit NID (Node ID) of the originator. In the second packet this has been effectively made implicit since FFFF is a reserved ID meaning 'auto insert my prime NID'. In fact both packets would have the same result except that if a transmission error occurred for the first packet then if the originator did not actually have an ID of 0010 it could not be solicited for a re-transmission. So second packet format is better.

        Comment

        • Borden1
          • Mar 2024

          #5
          Karam,

          Thanks for the detailed explanation.

          The issue looks to be one that highlighted

          Comment

          Working...
          X