Fun with Patents OR The Possible Future of Amazon Alexa and Google Home

In the article Hey, Alexa, What Can You Hear? And What Will You Do With It?, The New York Times delved into some of the patents that Amazon and Google have filed for the future of their voice assistants (Amazon Alexa and Google Home). The article focused on privacy concerns by the group Consumer Watchdog that may or may not have understood what a patent is. The stuff that really freaked people out was the Amazon patent that focused on an “always on” capability where the assistants are always listening to the discussions around them.

It’s an interesting idea to use the conversations in the room to develop a better understanding of them; however, the language used clearly doesn’t take privacy into account. The patent was filed more as a future idea rather than something with all the kinks figured out.  But I can understand why some phrases from the patent Keyword Determinations From Conversational Data upset people. To paraphrase:

In at least some embodiments, a computing device such as a smart phone or tablet computer can actively listen to audio data for a user, such as may be monitored during a phone call or recorded when a user is within a detectable distance of the device. In other embodiments, voice and/or facial recognition, or another such process, can be used to identify a source of a particular portion of audio content.

I thought some of the other patents might provide a window into how Amazon and Google viewed the future. My favorite one was titled Monitoring And Reporting Household Activities In The Smart Home According To A Household Policy and was written by Tony Fadell, founder of Nest and one of the fathers of iPod.

This patent talks about various different ways to make a home “smart.” Today having a smart home means being able to control various devices, but what if you could set a goal (or policy in the words of the patent) and the smart home would partner with you to achieve it. To paraphrase the language of the patent it is:

A method for household policy implementation in a smart home, comprising: monitoring the household, analyzing household activities, taking actions and reporting the information. This system can help a family achieve goals such as how much screen time is used by family members, how often the household eats together and whether mischief might be occurring.

Ignoring the obvious privacy issues, there were some interesting things here. As a father, this was really interesting because it thought of the way to install parental controls over my entire smart home.

Let’s start with the overall partnership model. As the parent, I get to define a goal and the house will help me achieve it. How will this work? Let’s look at the example of tracking screen time. I’m kind of excited about a future where I can say “Limit my kids to 30 minutes of screen time.”

First, we need to monitor screen time. We need to understand who is in the room and what they’re watching.

Then we need to define our goals.

Finally, we take an action based on whether the goal is met or not.

Other factors may come into play. For example, if the child has been grounded they may lose their TV time.

Also, just because this was pretty funny, I have to include the patent’s “mischief detector” that detects mischief by  (again paraphrasing):

listening for low-level audio signatures (e.g., whispering or silence), while the occupants are active (e.g., moving or performing other actions). Based upon the detection of these low-level audio signatures combined with active monitored occupants, the system may infer that mischief (e.g., activities that should not be occurring) is occurring. Additionally, contextual information such as occupancy location may be used to exclude an inference of mischief. For example, when children are near a liquor cabinet or are in their parents’ bedroom alone, the system may infer that mischief is likely to be occurring.

While I probably won’t be using the mischief tracker any time in the future, the idea of setting goals for the household, and letting Amazon and Google help, is quite appealing.