Cortex voice control using Amazon Echo

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts
  • spellinn
    Automated Home Sr Member
    • Dec 2007
    • 53

    Cortex voice control using Amazon Echo

    Having purchased an Amazon Echo now they are available in the UK, I was curious as to how difficult it would be to connect it to my existing Cortex system now we have a Web API available.

    After about four hours of reading/coding, here's an early working demo:



    Still very much a work in progress, and only supports the basic on/off commands so far, but it at least demonstrates it's possible to use Echo to decode the instruction (via the Amazon cloud), send it to my code in AWS Lambda which decodes the request and calls the Cortex web API back in my home network via my broadband connection. Note my Cortex installation is particularly slow in responding generally (even to module button presses) so don't take this demo as indicative of general performance.


    Cheers


    Neil
  • chris_j_hunter
    Automated Home Legend
    • Dec 2007
    • 1713

    #2
    well done ... am tempted to follow ...

    ultimately, perhaps with several Dots around the house ... 'though that seems to need future software updates, to avoid confusion ...

    wonder how effective / reliable when house is not quiet - others talking, radio or television on, washing machine whirring, etc ...

    Chris
    Last edited by chris_j_hunter; 11 October 2016, 10:05 AM. Reason: typo
    Our self-build - going further with HA...

    Comment

    • spellinn
      Automated Home Sr Member
      • Dec 2007
      • 53

      #3
      Originally posted by chris_j_hunter View Post
      wonder how effective / reliable when house is not quite - others talking, radio or television on, washing machine whirring, etc ...
      Chris
      I've been impressed with the recognition so far. "She" has a good ability to cancel out background noise, and the accuracy of recognition seems to be the same with the TV on or music playing to a quiet room.

      Cheers

      Neil

      Comment

      • Karam
        Automated Home Legend
        • Mar 2005
        • 863

        #4
        Originally posted by spellinn View Post
        Note my Cortex installation is particularly slow in responding generally (even to module button presses) so don't take this demo as indicative of general performance.
        Just so that the general audience don't get the wrong impression .... This is not the normal expectation of Cortex reaction speed and particularly so for lighting control. So I had already quizzed Neil about what might be causing this delay and he explained that he thinks its partly because his Cortex computer 'is somewhat overloaded with other stuff'.

        Comment

        • spellinn
          Automated Home Sr Member
          • Dec 2007
          • 53

          #5
          Originally posted by Karam View Post
          Just so that the general audience don't get the wrong impression .... This is not the normal expectation of Cortex reaction speed and particularly so for lighting control. So I had already quizzed Neil about what might be causing this delay and he explained that he thinks its partly because his Cortex computer 'is somewhat overloaded with other stuff'.
          Whilst my PC isn't ideal, the lamps in the video are actually slower than all the other lights in my house as they are all on a RTX-401 wireless module, and switching the plugs probably takes 1-2 seconds longer to respond than the light modules wired to traditional DIN dimmers.

          Cheers

          Neil

          Comment

          • chris_j_hunter
            Automated Home Legend
            • Dec 2007
            • 1713

            #6
            presumably the Echo working via the cloud contributes to the delay, too ... ?
            Our self-build - going further with HA...

            Comment

            • spellinn
              Automated Home Sr Member
              • Dec 2007
              • 53

              #7
              Originally posted by chris_j_hunter View Post
              presumably the Echo working via the cloud contributes to the delay, too ... ?
              Yes, the Echo takes about 2-3 seconds to decode anything you say to it..so add on the 2-3 seconds for the wireless plug sockets and you get the performance you see in the video.

              I will do some more accurate timings once more of the code is written to adjust temperature and dimming levels.

              Cheers

              Neil

              Comment

              • chris_j_hunter
                Automated Home Legend
                • Dec 2007
                • 1713

                #8
                'tis funny, initial reaction to 2-3 seconds delay is that it's too much ... but asking someone, a person, to do the job would probably involve the same, perhaps more ... difference, though, is that perhaps feedback is more immediate, because the person would soon say something and/or show signs of action ... maybe the Echo does, too, via an LED ?
                Our self-build - going further with HA...

                Comment

                • spellinn
                  Automated Home Sr Member
                  • Dec 2007
                  • 53

                  #9
                  Originally posted by chris_j_hunter View Post
                  'tis funny, initial reaction to 2-3 seconds delay is that it's too much ... but asking someone, a person, to do the job would probably involve the same, perhaps more ... difference, though, is that perhaps feedback is more immediate, because the person would soon say something and/or show signs of action ... maybe the Echo does, too, via an LED ?
                  Yes, for me, the delay isn't a problem, as they way you talk to the Echo is usually whilst you're performing other tasks, or have your hands full, so you don't really sit there waiting for a response.

                  There is a LED which rotates whilst she is "thinking" - you can just make this out in the video..it's just below and to the left of the clock on the wall near the picture frames.

                  Cheers

                  Neil

                  Comment

                  • chris_j_hunter
                    Automated Home Legend
                    • Dec 2007
                    • 1713

                    #10
                    yep, think we will take this onboard - will wait for the multi- update ...

                    there's also a Google option, and of course Apple Siri, but no idea how they compare, especially from a linkability to Cortex point of view ... plus TVs are beginning to be spoken-to, too, so that's another factor to consider ... maybe several of these things can co-exist, like people can !

                    funny how things develop - when we first planned our set-up, we didn't include intercom abilities, and yet we now wouldn't dream of going without, not only for intercom functions, but also for Cortex telling us things ... ditto, since Karam noted that low-cost Android tablets offered possibilities, and we brought them on-board, too (now having seven, and may yet go for more) ...

                    security is perhaps a concern ... most graphically since yesterday I noticed this :

                    A powerful cyber-attack came close to destroying a French TV network, its director-general tells the BBC.


                    Chris
                    Last edited by chris_j_hunter; 11 October 2016, 01:34 PM.
                    Our self-build - going further with HA...

                    Comment

                    • chris_j_hunter
                      Automated Home Legend
                      • Dec 2007
                      • 1713

                      #11
                      Neil -

                      just had a quick look into AWS Lambda, via ...

                      Lambda is a compute service that lets you build applications without provisioning or managing servers.


                      did your four hours or so include getting to grips with it for the first time, as a novice, or were you already into it ... ? ?

                      just trying to gauge how long we might need !

                      last experience of such things was with the Met.Office DataPoint facility - which took a while, especially the overlay side of things, 'though seemed all v.easy once did get into it ...

                      Chris
                      Last edited by chris_j_hunter; 11 October 2016, 03:05 PM.
                      Our self-build - going further with HA...

                      Comment

                      • spellinn
                        Automated Home Sr Member
                        • Dec 2007
                        • 53

                        #12
                        Originally posted by chris_j_hunter View Post
                        Neil -

                        just had a quick look into AWS Lambda, via ...

                        Lambda is a compute service that lets you build applications without provisioning or managing servers.

                        did your four hours or so include getting to grips with it for the first time, as a novice, or were you already into it ... ? ?
                        It was my first time, plus I've never programmed in Python before (I do know C, PHP etc), so there's been a small learning curve on the syntax differences. The Echo SmartHome SDK provides some Lamba templates you can use to get going quickly.

                        Once I have something resembling a fully working skill I'm happy to upload the code to Github so others can reuse it.

                        There will be some changes that Idratek will need to make to their API support for it to be provided as a downloadable "marketplace" skill for Echo..mainly allowing Oath2 authentication tokens against the WebAPI, plus a way of registering your API endpoint address so the Lambda code knows where to send the API calls.

                        It's still very early days though.

                        Cheers

                        Neil

                        Comment

                        • chris_j_hunter
                          Automated Home Legend
                          • Dec 2007
                          • 1713

                          #13
                          that would be good ...

                          we've learnt that interacting with Cortex in various ways is a good thing - flexibility, for freedom of action in various circumstances ...

                          showed your video to SWMBO, and the BBC one on 'Echo, and got a positive response ...

                          Amazon is bringing its range of voice-controlled Echo speakers to the UK and Germany.


                          Chris
                          Our self-build - going further with HA...

                          Comment

                          • chris_j_hunter
                            Automated Home Legend
                            • Dec 2007
                            • 1713

                            #14
                            Neil -

                            any news, any more successes ?

                            have been watching the EvoHome / Echo thread in the Heating Control forum - seems less than smooth, 'though perhaps for a range of reasons ...

                            Chris
                            Our self-build - going further with HA...

                            Comment

                            • spellinn
                              Automated Home Sr Member
                              • Dec 2007
                              • 53

                              #15
                              Originally posted by chris_j_hunter View Post
                              Neil -

                              any news, any more successes ?

                              have been watching the EvoHome / Echo thread in the Heating Control forum - seems less than smooth, 'though perhaps for a range of reasons ...

                              Chris
                              Decided to resurrect my initial PoC code as we now have an Echo on every floor so voice control of Cortex is becoming an increasing nice to have.

                              Amazon have made some useful additional to the Smarthome API since I last looked at this which will simplify the bridge between the Amazon and Cortex APIs.

                              Will hopefully post back some useful working code here at some point.

                              Regards

                              Neil

                              Comment

                              Working...
                              X