Eavesdropping on private conversations is one of the most common and unpleasant privacy threats. It's not for nothing that many people glue microphones and cameras on laptops to keep the most personal and private moments a secret. Well, it's time to glue the lidars on your smart vacuum cleaner as well. Because scientists from the National University of Singapore have found a way to wiretap rooms with a vacuum cleaner.
Recently, researchers have proposed several methods of wiretapping through various smart devices of the "smart home". This paper describes LidarPhone - a new acoustic attack channel through lidars, which are equipped with popular home robotic vacuum cleaners. The main idea is to repurpose the lidar to work as a laser microphone that can pick up sounds from subtle vibrations induced on nearby objects.
Sounds are pressure waves that propagate through the vibrations of the medium. Consequently, sound energy in the environment is partially induced on nearby objects, creating subtle physical vibrations within these solid media. The fundamental concept of LidarPhone is to sense such induced vibrations in household items using the lidar sensor of the robot vacuum cleaner, and then process the recorded vibration signal to reconstruct sound traces. This sensing method is inspired by the principle of laser microphones, which use reflected laser beams to pick up sounds from vibrating objects.
How an inexpensive lidar works
Although laser microphones require complex adjustments, rotating lidar sensors are equipped with at least a laser transmitter and a reflection sensor. This provides key capabilities for converting lidar to microphone. Of course, the practical implementation of this idea requires overcoming a number of difficulties.
One of the main problems when using lidar as a laser microphone is the extremely low signal-to-noise ratio (SNR) of reflected signals. This is partly due to the fact that LidarPhone relies on different physics principles than laser microphones, despite their obvious similarities at a high level. Laser microphones target highly reflective materials (i.e., producing specular reflections), such as high SNR glass windows. In contrast, hardware lidar amplifiers and analog-to-digital converters (ADCs) are tuned to be sensitive only to low-intensity signals that are mostly reflected off rough surfaces such as walls, thereby producing diffuse reflections. Therefore, even if the lidar receives high intensity signals from the glass window,he will not receive useful information.
Difference between directional specular reflection for a laser microphone (left) and diffuse reflections with lower SNR received by LidarPhone
To overcome the problem of low SNR, scientists have used various signal processing techniques, including filtering out frequency components that contain noise. To further reduce the effect of noise, they performed noise reduction by subtracting dynamically profiled noise using spectral analysis. In more detail, physical and mathematical methods are described in scientific work .
Another major issue in the development of the LidarPhone attack was the low sampling rate of the lidar. Given its rotational motion, the sampling rate for one point on the target is equivalent to the rotation rate of the lidar. The scientists increased the sampling rate by multiplying the lidar speed (5 Hz) by the number of samples per revolution (360). Thus, the sampling frequency has increased from 5 Hz to 1.8 kHz, which is an important indicator when training a neural network.
LidarPhone carefully processes and extracts audio signal traces from initially noisy laser reflections. In this way, confidential information is captured (for example, speech emitted by a computer speaker when the victim is participating in a teleconference).
In this scientific paper, as an experiment, the attack was tested on the Xiaomi Roborock robot vacuum cleaner. The LidarPhone prototype was trained on numbers from computer speakers and music from a TV speaker: more than 30 thousand audio samples with a total duration of more than 19 hours of recorded audio. As a result, LidarPhone showed an average accuracy of about 91% and 90% when classifying numbers and music, respectively.
In the experiment, LidarPhone received confidential speech information from laser beams reflected from the smallest vibrating objects, such as a trash can or a plastic bag. The illustration below shows the appearance of a sound signal with reflection from various objects.
The figure corresponds to the sounds in the word "nine" derived from a) the original audio; b) microphone recording from a distance of 3 meters; c) processed trashcan lidar records; d) IKEA package. Corresponding sound files are published at bit.ly/lidarphone-sensys
LidarPhone overcomes the limitations of modern wiretapping methods that require some kind of eavesdropping equipment to increase the chances of detection. There are no bugs or extraneous equipment here. Only an ordinary vacuum cleaner that stands modestly in the corner and does not arouse suspicion.
The researchers write that many other light sensors, including those on smartphones, could theoretically be used as microphones. Recently, a number of scientific papers have uncovered use cases for non-acoustic sensors in smartphones that potentially allow you to record conversations without permission. We are talking about the use of an accelerometer, gyroscope and vibration motor ( 1 , 2 , 3 , 4 ).
But using a smartphone for wiretapping is trivial and predictable. Home appliances with lidars, which no one can suspect of eavesdropping on conversations, is a completely different matter. Such as a vacuum cleaner. In general, the proliferation of smart devices in our homes opens up many opportunities for this type of acoustic attacks by eavesdropping on private conversations. The main thing to remember is that a lidar is actually a ready-made laser microphone.
Research paper released on November 16, 2020 at SenSys '20 Virtual Conference in Japan.
