Don’t cough near your Amazon Echo. Keep your heart rate steady and your palms dry while pushing your Walmart shopping cart. Just maintain normal, unsuspicious homeostasis until you can sneak away from your devices and build a quiet life for yourself in the forest.
On Tuesday, Amazon secured a patent for technology that would let its artificial intelligence assistant Alexa analyze people’s voices for signs of illness and use those cues to sell them products. The same day, Amazon competitor Walmart secured a patent for an interactive display that can sync up with another pending Walmart patent: a shopping cart handle that can record a customer’s biometric data. Those moves come weeks after life insurance giant John Hancock announced that all its plans would encourage customers to wear an activity tracker like a FitBit.
Big Brother is watching your vital signs, and using them to sell you things.
ADVERTISEMENT
Patents like Amazon’s and Walmart’s aren’t a guarantee that the companies will make those products, but the patents mean the companies have given those ideas considerable thought.
Amazon’s patent describes a voice-recognition system that could identify signs of an ailing Amazon Echo owner, the Telegraph first reported. The Echo, Amazon’s smart home device, currently uses Alexa software to listen for commands, including requests to play music or order products from Amazon. But soon Alexa might listen to more than what you say — it’ll listen to how you say it.
“Alexa, *cough* I’m hungry *sniffle*,” Amazon’s patent describes a hypothetical user as saying. If Amazon rolls out its newly patented software, Alexa will be able to recognize the sniffles and respond how it knows best: recommending products.
“The voice interaction device may present the audio content,” the patent reads. “The selected content may be an offer to purchase or order cough drops with expedited delivery. The content may be selected based at least in part on the likelihood that the user has a sore throat and/or the physical or emotional abnormality detected in the user's voice input.”
Alexa might also recommend things like chicken soup recipes. Some customers might be unnerved by a live-in robot scanning their voices for signs of weakness. In that case, if they turn down chicken soup, the Alexa software is programmed to offer them other remedies. “OK, I can find you something else,” the patent describes Alexa as saying after a customer declines soup.
Amazon describes the hypothetical customer relenting and letting Alexa order cough drops with one-hour delivery. The technology could also be tailored to detect a user’s mood, which could also be used to sell things, according to the patent.
“In one example conversation, a user may say ‘Alexa, tell me joke.’ The voice interaction device or a connected computer system may determine that the user is sleepy and bored,” the patent reads. Advertisers could promote products, like specific music, for sleepy and bored users. “For example, the voice interaction device may audibly present ‘here's a joke [ . . . ] By the way, this singer just released his new album for just $1.99. Do you want to preview it?’”
Meanwhile, Walmart is considering measuring customers’ health and moods through their skin. Their pending application describes a shopping cart handle with built-in biometric readers that track body temperature, walking speed, grip, and heart rate, Motherboard previously reported.
The goal is “identifying customers who may be in need of customer service, and more specifically to identifying those customers based on biometric feedback collected from their shopping cart handle,” according to the patent application.
The cart will study the customer’s vitals and make predictions as they shop.
“The server can, over time, build a table of the data associated with a customer's visit to the store, the table being made of the biometric data and/or the non-biometric cart data,” and make extrapolations about the customer, according to the patent application. “For example, the server can create, within the table, a metric for the customer's stress. This stress estimate can, for example, be calculated by weighting the biometric and non-biometric factors. As one example, if a customer's temperature is increasing while the customer's grip on the shopping cart handle simultaneously increases in force, the stress estimate may increase.”
If a customer is moving erratically, the cart might dispatch an employee to check on them, the application describes. It also suggests compiling biometric data across carts to determine customers’ least-favorite areas in the store.
Walmart is also considering syncing the handle with “an interactive display,” which sells things. On Tuesday, the company secured a patent for the system, which would allow customers to save their shopping preferences and credit card information, or add more items to a digital shopping cart. Customers would control the display through buttons in their shopping cart handle.
With this system in place, the handles would be able to collect even more personally identifying biometric data. To log into their Walmart accounts on the interactive display, “the customer can provide biometric information using the biometric device on the shopping cart,” the patent reads. “The biometric device can detect one or more of the following biometric properties: finger prints, facial images, iris prints and/or voice recognition.”Storing personal information like fingerprints appears to conflict with Walmart’s description of the cart handles in their patent application.
“It is noted that the biometric data and the cart movement data collected during the use of the shopping cart is not tied or otherwise linked to the identity of the individual customer,” the pending application reads. “When carts are idle, before being selected by a customer for the shopping experience, the cart contains no information about actual or potential customers.”
Even when companies pledge to keep user data private, the sensitive information has a way of leaking. Amazon’s Alexa products have been plagued by allegations of the devices recording users without their knowledge, even sending those recordings to random contacts in the case of an Oregon woman earlier this year.
And millions of Walmart customers have had their personal information exposed in a series of data breaches, including a string of massive cyber attacks in 2005 and 2006. A 2015 hacking attack exposed Walmart Canada customers’ information, including their credit card numbers. In March, one of Walmart’s partner companies accidentally exposed 1.3 million Walmart customers by uploading unsecured files titled “walmartsql” to Amazon’s cloud storage platform
The files “contained customers’ names, addresses, ZIP codes, phone numbers, email addresses, IP addresses, plain-text passwords, encrypted credit card numbers, and payment details for purchases made between 2000 and early 2018.”
At the time, those were the only details Walmart had collected on customers. In the future, Walmart might store their fingerprints.
