This article is more than 1 year old

Alexa, am I having a heart attack? Here's how smart speakers could detect their masters spluttering to death

Urgh, OK Google, call 9... Can you hear me?! CALL 9...arghgh

Digital assistants such as Amazon Alexa and Google Home may be able to automatically detect when someone is having a heart attack, and call for medical help, one day, according to this latest research.

Smart speakers are always listening for wake words, and recording everything they hear to improve their neural networks and target their masters with marketing. It's, frankly, creepy. Academics at the University of Washington in Seattle, USA, however, believe there are benefits to installing a cyber-assistant at home that listens in all the time.

“A lot of people have smart speakers in their homes, and these devices have amazing capabilities that we can take advantage of,” said Shyam Gollakota, coauthor of the research published this week in npj Digital Medicine.

“We envision a contactless system that works by continuously and passively monitoring the bedroom for an agonal breathing event, and alerts anyone nearby to come provide CPR. And then if there’s no response, the device can automatically call 911.”

The system is based on a type of machine-learning algorithm known as support vector machine (SVM). It can be trained to listen out for the wheezing and gasping associated with shortness of breath during heart attacks through the microphones installed in digital assistant gadgets.

The researchers trained their algorithm on a dataset of real 911 calls made to Seattle's emergency services between 2009 and 2017. They analysed the 162 cardiac arrest calls and took 236 clips each lasting about 2.5 seconds.

To boost the dataset further, they played the recordings over distances of one, three, and six metres in indoor and outdoor surroundings at different volumes, and recorded these using three devices. One was an Amazon Alexa, the other was an iPhone 5s, and the third was a Samsung Galaxy S4 handset. The idea was to increase the range of situations in which a legit emergency call could be made. They thus generated and collected a total of 7,316 samples of people experiencing heart attacks, and trained the SVM on the trove of audio.

The SVM was also trained on a negative dataset, also known as the sleep dataset, which contains 7,305 audio samples of other types of noises that aren’t associated with heart attacks. Sounds from snoring, breathing in sleep, and background noises common in bedrooms like podcasts or white noise were included so that the SVM algorithm can discern between real cardiac arrest sounds and false positives.

Oops, we need noise cancellation for this to work

When the researchers tested their system on samples from the positive dataset, the devices correctly identified the noises associated with heart attacks about 96 per cent of the time when they were placed six metres away from the source of the sound. But there’s a catch: it only really works when the researchers apply a noise cancellation algorithm to drown out background noise.


Boffins' neural network can work out from your speech whether you'll develop psychosis


The percentage for identifying heart attacks rapidly drops to just below five per cent if a noise cancellation algorithm isn't applied. In other words, the machines probably can’t hear you croak in pain if the TV or if your Roomba robot is humming away and the device recording you splutter your final breath can't compensate for this distraction.

Here's some good news: the false positive rate was pretty low at about 0.14 per cent when the SVM model was tested on over 82 hours of material gathered from the sleep data. That figure rose to 0.22 per cent when the researchers tested their algorithm on 164 hours of real data recorded in 35 different bedrooms.

“Right now, this is a good proof of concept using the 911 calls in the Seattle metropolitan area,” Gollakota said. “But we need to get access to more 911 calls related to cardiac arrest so that we can improve the accuracy of the algorithm further and ensure that it generalizes across a larger population.”

Despite the mediocre results – the SVM doesn't work without top-notch noise cancellation – the boffins hope to improve the model further so that they can commercialize the technology. Gollakota believes that the SVM can be packaged into an app or as a feature built into digital assistants like Amazon Alexa or Google Home.

“This could run locally on the processors contained in the Alexa. It’s running in real time, so you don’t need to store anything or send anything to the cloud,” he added. ®

More about


Send us news

Other stories you might like