Researchers eavesdrop on smartphone finger taps

Researchers have been experimenting with a novel way to eavesdrop on what you’re typing on your smartphone: They listen to the interaction between your fingers and the screen.

In a paper called Hearing your touch: A new acoustic side channel on smartphones, academics at the University of Cambridge claim to demonstrate…

The first acoustic side channel attack that recovers what users type on the virtual keyboard of their touch-screen smartphone or tablet.

In tests, they showed moderate success guessing what users entered into the software-based keyboard devices.

According to the researchers’ paper, when users tap out numbers and letters on a smartphone’s virtual keyboard, it generates a sound wave that travels both on the surface of the screen and in the air. The location of the user’s finger on the screen distorts the wave, and the microphone on the mobile device picks up this distortion, enabling an algorithm to “hear” what the user typed.

To make it work, the researchers use machine learning (a form of artificial intelligence) to train their algorithm using around 21 hours of audio recordings of finger taps.

They tried out the attack with 45 participants in a real-world environment, using both an Android tablet and an Android smartphone. On the smartphone, they tried listening to four-digit PIN codes and were able to retrieve the correct codes 61% of the time within 20 attempts.

On the tablet, they successfully retrieved 9 codes between 7 and 13 characters in length in 50 attempts.

Should you be worried about this?

Probably not. For it to work, an attacker would have to install malware on your device. Then, you would have to give it access to the device’s microphone. And if an attacker is running malware on your smartphone, you’ve got a lot more to worry about than whether it can listen to what you’re typing.

The speed of sound in air depends on temperature, meaning that the results might be skewed if it’s too hot or too cold (the researchers only tested in a relatively tight temperature window because of this). They also said that an additional glass layer on top of the screen could absorb most of the finger impact, making an effective countermeasure. User-installed protective glass and plastic coatings are pretty common on phones now to protect those increasingly large screens. Less so on tablets, though, admittedly.

All in all, this was an interesting intellectual exercise, but there are easier ways to slurp a target’s data. It might make for a fun scene in a Hollywood movie with a credulous audience, though.