04-10-2017, 03:45 PM
Once again I'm starting a new thread on this, partly because yet again a lot of water has passed under the bridge and I don't want newly interested folks to have to wade through it. And, partly due to a name change before it goes public. Turns out that there is a trademark for Jarvis in the context of digital assistants, by of all people Marvel. It is apparently for mobile devices, which doesn't apply to our stuff. But, just to avoid any potential issues, the name will become CQCVoice as of 5.0.914 and forward. That's generic enough that it shouldn't cause any issues.
---------
The upcoming 5.1 release will contain a new feature, called CQCVoice, which is an all-local voice control mechanism. All-local meaning it is not cloud based like the Echo, it's working totally locally within your system. Both the Echo and Voice have their own pros and cons, and I will try to lay them out logically here so that folks can make a reasonable comparison. But first a little background...
We've supported the Echo for some time, and it works well with CQC now that it's been sorted out over time. But, it is cloud based, which means that is subject to disruption, and it also has its security implications because it requires a connection back into your home network from the Amazon servers. Some folks are more concerned with either of those than others, and some very concerned about both. It also requires a considerable effort to set up.
So we wanted to have a local, easier option. This involved quite a dive into a new world of speech recognition, audio data processing, command and control grammars (more about that below), and microphone hardware. The first few rounds were a bit disappointing, but good old desperation and inability to spell defeet has finally allowed us to make a lot of progress and it's getting pretty sorted out now as well. It's one of those systems that could be indefinitely expanded, but it's getting pretty close to something that is ready for an inaugural release.
I don't want do duplicate the help information, but for a little semi-technical background here... 
Types of Voice Recognition
There are roughly two types of voice recognition. There is dictation and there is command and control. The former almost always requires training and only works for people who have trained for it, and the program has to know who is talking. And you typically have to speak sort of slowly with a solid division between words, because the program has to recognize any possible word you might speak, a freakishly difficult task. Command and control style recognition is based on a limited 'grammar', which defines what phrases can be recognized.
Both the Echo and CQCVoice are of the latter type, though the Echo combines a little of both. With the Echo we can define a phrase like:
The Echo can be told that {whatever} can be from 1 to x number of words, and they can be any words. But the rest of the phrase has to be fixed, though variations can be defined. So the Echo is recognizing small numbers of arbitrary words, which is sort of dictation-like. It is able to do this without training by basically having enormous computing resources available to it on its servers, and reams of memory, and probably quite advanced recognition algorithms. Though, even so, it still gets things wrong sometimes.
CQCVoice is a pure command and control grammar, in that all of the phrases have to be predefined. However, that doesn't mean that the grammar cannot be created on the fly and then compiled, and that is what we do. We have the basic grammar, which we load, and then we have rules in that grammar that are placeholders into which we can plug the names of your lights, your playlists, your security zones, and so forth. So it's fixed in one sense, but still somewhat dynamic.
Still, once we've done that and rebuilt the grammar, those are the only phrases that are recognized. So, unlike above, in the Echo case, where Amazon will hand us back whatever text it thought was spoken for {whatever}, all CQCVoice can do is tell us the fixed phrase in the grammar that it thought was matched, along with confidence level information to help us make decisions about how trustworthy that match is. 
This is also a security 'feature' in that we cannot hear what you are saying. We can only be told what phrase matched closest to what you are saying. So if you are just talking in the room, we'll just get a string of very low confidence matches that we'll just ignore. 
Ease of Setup
One huge advantage of CQCVoice is the ease of setup. Since it's all local, there's no setup of any servers or port forwarding or security certificate stuff involved as there is with the Echo. And, since it is based on the Room/System configuration data that you may already be using to auto-generate templates, basically you sort of get it for free. You just have to make some simple configuration changes to the room configuration to associate a CQCVoice host with a room. The CQCVoice program on that machine looks through the rooms for one that has his host name configured and so he knows that is his room.
Of course the down side is that you mostly only have control over those things that are defined by the system/room configuration. We will now obviously want to expand this over time, but still it's not as open ended as the Echo is, where you have more control over the operations involved and what is controllable. There is a little open ended'ness currently, in that you can define a set of 'room modes', which are just global actions that you can invoke via voice control.
It also means we cannot have a command like "Zira, remind me in 10 minutes to do X", because we would have to define every possible variation of X. We could, and probably will, allow you to define such a list for yourself, so there are always ways to get around it. But we cannot get the arbitrary words you spoke for X and then later say, "You ask me to remind you to do X." We have to have a defined set of things for X that can be matched. That obviously make it less spontaneously useful.
Another advantage to CQCVoice is that it is more conversational. The Echo is really just a voice driven remote control, which has no idea what you are talking about, it's just matching phrases. CQCVoice does know what you are talking about, so it can ask you for clarifications, allow you to implicitly refer to things you were just talking about, and be more interactive in general. It knows that some pieces of information you say are more important than others, and can be safer by requiring higher confidence matches for those things, and lower for less critical things.
I will admit that what is actually possible is not as nice as what I'd originally hoped for. My original vision had CQC/CQCVoice winning the Presidential Courage award for calling emergency services on behalf of house members and talking the kids through medical procedures via conversation with emergency personal while simultaneous guiding them to the home and relaying vital statistics. 
Sadly this is beyond the means of any reasonable voice recognition system we could afford to use. The problem becomes ambiguity in the grammar. You can only support so many phrases and variations of phrases before you drown in ambiguity and the accuracy of recognition drops substantially. 
So we have cut the grammar down from the original form. But still, it is quite conversational and can guide you to getting the operation done correctly, unlike the Echo which either gets it right or not. We didn't reduce the number of commands, just reduced the 'floweriness' of the available phrases you can say.
It also means that you can invoke multiple commands in a row without having to constantly say the key word. Once you start a conversation, you can continue to just invoke commands in a more natural manner. When you are done, you can say things like "That will be all" or "That's all for now" or "You are dismissed", as you would say to dismiss a real butler. If you don't say anything for 10'ish seconds, it will ask if you want anything else, to give you a chance to continue the conversation. If you say nothing still it will dismiss itself.
Hardware
The Echo obviously is its own hardware. CQCVoice can use any microphone that can be configured to show up as the recording input in Windows. There are limitations of course. For distant recognition, you really need a microphone array type product. These, using fancy signal processing algorithms, allow the device to create a 'virtual directional' microphone. I.e. it can hear where the loudest sound is coming from, and selectively emphasize sound from that area, rejecting 'off axis' noise. This greatly improves recognition accuracy from a distance.
The Microsoft Kinect 2 is such a product, and one of the less expensive ones, though it also includes various other features that we aren't making use of. It's too bad they don't split out just the microphone array part. There are also various, more expensive to varying degrees, array mics designed mostly for the conference room market. These tend to be larger and have more mics than the Kinect, which means better reception and more off axis rejection. Some of them also have nice signal processing software on board that can do reverb cancellation if you have a more echo'y room.
One such company is Acoustic Magic, which have the Tracker 1 and Tracker 2 products, for non-crazy prices. The 1 version is the one you'd want for this application. The 2 has other features that would not be useful for CQCVoice. Of course we cannot guarantee any particular performance, not having been in a position to try one ourselves, so make sure you can return any such products if they don't perform as expected. This is just one company that we've found that is selling a reasonably priced mic array product the specs of which would seem to indicate an real advantage over the Kinect.
Others can get quite pricey, though they probably also have substantially increased performance as well, and might have some placement advantages. I.e. some are roundish and designed to sit on a table top, which might be optimal for a living room based setup (assuming it won't be subject to serious abuse in such a setting.) Others are 'linear' arrays that are long and skinny and designed to mount on the wall, which may also keep them out of harm's way better.
For close up work, if you wanted to do voice control while sitting at your desk working on our computer, a small desktop microphone or a headset would likely work just fine. The front of the computer mic might work just fine as well when you are sitting right there in front of it.
Anyway, the moral of the story is, we've now done away with the Kinect requirement, so it's up to you what you want to use. You can try anything you want and see if it works. 
Summary
So, that is it in a nutshell. It's coming along nicely, and folks are now reporting good accuracy, as long as you use the appropriate hardware, and don't drink too much before issuing commands (IVB this means you.) There are recognition files for various English accents as well, which can help.
---------
The upcoming 5.1 release will contain a new feature, called CQCVoice, which is an all-local voice control mechanism. All-local meaning it is not cloud based like the Echo, it's working totally locally within your system. Both the Echo and Voice have their own pros and cons, and I will try to lay them out logically here so that folks can make a reasonable comparison. But first a little background...
We've supported the Echo for some time, and it works well with CQC now that it's been sorted out over time. But, it is cloud based, which means that is subject to disruption, and it also has its security implications because it requires a connection back into your home network from the Amazon servers. Some folks are more concerned with either of those than others, and some very concerned about both. It also requires a considerable effort to set up.
So we wanted to have a local, easier option. This involved quite a dive into a new world of speech recognition, audio data processing, command and control grammars (more about that below), and microphone hardware. The first few rounds were a bit disappointing, but good old desperation and inability to spell defeet has finally allowed us to make a lot of progress and it's getting pretty sorted out now as well. It's one of those systems that could be indefinitely expanded, but it's getting pretty close to something that is ready for an inaugural release.
I don't want do duplicate the help information, but for a little semi-technical background here... 
Types of Voice Recognition
There are roughly two types of voice recognition. There is dictation and there is command and control. The former almost always requires training and only works for people who have trained for it, and the program has to know who is talking. And you typically have to speak sort of slowly with a solid division between words, because the program has to recognize any possible word you might speak, a freakishly difficult task. Command and control style recognition is based on a limited 'grammar', which defines what phrases can be recognized.
Both the Echo and CQCVoice are of the latter type, though the Echo combines a little of both. With the Echo we can define a phrase like:
Code:
Set the {whatever} to bongo modeThe Echo can be told that {whatever} can be from 1 to x number of words, and they can be any words. But the rest of the phrase has to be fixed, though variations can be defined. So the Echo is recognizing small numbers of arbitrary words, which is sort of dictation-like. It is able to do this without training by basically having enormous computing resources available to it on its servers, and reams of memory, and probably quite advanced recognition algorithms. Though, even so, it still gets things wrong sometimes.
CQCVoice is a pure command and control grammar, in that all of the phrases have to be predefined. However, that doesn't mean that the grammar cannot be created on the fly and then compiled, and that is what we do. We have the basic grammar, which we load, and then we have rules in that grammar that are placeholders into which we can plug the names of your lights, your playlists, your security zones, and so forth. So it's fixed in one sense, but still somewhat dynamic.
Still, once we've done that and rebuilt the grammar, those are the only phrases that are recognized. So, unlike above, in the Echo case, where Amazon will hand us back whatever text it thought was spoken for {whatever}, all CQCVoice can do is tell us the fixed phrase in the grammar that it thought was matched, along with confidence level information to help us make decisions about how trustworthy that match is. 
This is also a security 'feature' in that we cannot hear what you are saying. We can only be told what phrase matched closest to what you are saying. So if you are just talking in the room, we'll just get a string of very low confidence matches that we'll just ignore. 
Ease of Setup
One huge advantage of CQCVoice is the ease of setup. Since it's all local, there's no setup of any servers or port forwarding or security certificate stuff involved as there is with the Echo. And, since it is based on the Room/System configuration data that you may already be using to auto-generate templates, basically you sort of get it for free. You just have to make some simple configuration changes to the room configuration to associate a CQCVoice host with a room. The CQCVoice program on that machine looks through the rooms for one that has his host name configured and so he knows that is his room.
- There is also new 'spoken name' configuration for things like lights and security zones. You may have given a light the name MBR NiteLt, but you'll never match that when speaking. So, if the actual title is of that non-spoken form, you can now also provide an 'as spoken' name that will be used by CQCVoice.
Of course the down side is that you mostly only have control over those things that are defined by the system/room configuration. We will now obviously want to expand this over time, but still it's not as open ended as the Echo is, where you have more control over the operations involved and what is controllable. There is a little open ended'ness currently, in that you can define a set of 'room modes', which are just global actions that you can invoke via voice control.
It also means we cannot have a command like "Zira, remind me in 10 minutes to do X", because we would have to define every possible variation of X. We could, and probably will, allow you to define such a list for yourself, so there are always ways to get around it. But we cannot get the arbitrary words you spoke for X and then later say, "You ask me to remind you to do X." We have to have a defined set of things for X that can be matched. That obviously make it less spontaneously useful.
- I'll use Zira as the keyword in any examples here, but this is configurable so you can call it what you want.
Another advantage to CQCVoice is that it is more conversational. The Echo is really just a voice driven remote control, which has no idea what you are talking about, it's just matching phrases. CQCVoice does know what you are talking about, so it can ask you for clarifications, allow you to implicitly refer to things you were just talking about, and be more interactive in general. It knows that some pieces of information you say are more important than others, and can be safer by requiring higher confidence matches for those things, and lower for less critical things.
I will admit that what is actually possible is not as nice as what I'd originally hoped for. My original vision had CQC/CQCVoice winning the Presidential Courage award for calling emergency services on behalf of house members and talking the kids through medical procedures via conversation with emergency personal while simultaneous guiding them to the home and relaying vital statistics. 
Sadly this is beyond the means of any reasonable voice recognition system we could afford to use. The problem becomes ambiguity in the grammar. You can only support so many phrases and variations of phrases before you drown in ambiguity and the accuracy of recognition drops substantially. 
So we have cut the grammar down from the original form. But still, it is quite conversational and can guide you to getting the operation done correctly, unlike the Echo which either gets it right or not. We didn't reduce the number of commands, just reduced the 'floweriness' of the available phrases you can say.
It also means that you can invoke multiple commands in a row without having to constantly say the key word. Once you start a conversation, you can continue to just invoke commands in a more natural manner. When you are done, you can say things like "That will be all" or "That's all for now" or "You are dismissed", as you would say to dismiss a real butler. If you don't say anything for 10'ish seconds, it will ask if you want anything else, to give you a chance to continue the conversation. If you say nothing still it will dismiss itself.
Hardware
The Echo obviously is its own hardware. CQCVoice can use any microphone that can be configured to show up as the recording input in Windows. There are limitations of course. For distant recognition, you really need a microphone array type product. These, using fancy signal processing algorithms, allow the device to create a 'virtual directional' microphone. I.e. it can hear where the loudest sound is coming from, and selectively emphasize sound from that area, rejecting 'off axis' noise. This greatly improves recognition accuracy from a distance.
The Microsoft Kinect 2 is such a product, and one of the less expensive ones, though it also includes various other features that we aren't making use of. It's too bad they don't split out just the microphone array part. There are also various, more expensive to varying degrees, array mics designed mostly for the conference room market. These tend to be larger and have more mics than the Kinect, which means better reception and more off axis rejection. Some of them also have nice signal processing software on board that can do reverb cancellation if you have a more echo'y room.
One such company is Acoustic Magic, which have the Tracker 1 and Tracker 2 products, for non-crazy prices. The 1 version is the one you'd want for this application. The 2 has other features that would not be useful for CQCVoice. Of course we cannot guarantee any particular performance, not having been in a position to try one ourselves, so make sure you can return any such products if they don't perform as expected. This is just one company that we've found that is selling a reasonably priced mic array product the specs of which would seem to indicate an real advantage over the Kinect.
Others can get quite pricey, though they probably also have substantially increased performance as well, and might have some placement advantages. I.e. some are roundish and designed to sit on a table top, which might be optimal for a living room based setup (assuming it won't be subject to serious abuse in such a setting.) Others are 'linear' arrays that are long and skinny and designed to mount on the wall, which may also keep them out of harm's way better.
For close up work, if you wanted to do voice control while sitting at your desk working on our computer, a small desktop microphone or a headset would likely work just fine. The front of the computer mic might work just fine as well when you are sitting right there in front of it.
Anyway, the moral of the story is, we've now done away with the Kinect requirement, so it's up to you what you want to use. You can try anything you want and see if it works. 
Summary
So, that is it in a nutshell. It's coming along nicely, and folks are now reporting good accuracy, as long as you use the appropriate hardware, and don't drink too much before issuing commands (IVB this means you.) There are recognition files for various English accents as well, which can help.
Dean Roddey
Explorans limites defectum
Explorans limites defectum
