Scientists created Lamphone: using a photodiode and telescope, researchers turned bulbs into “bugs” for wiretapping

If you are impressed by the originality of Lev Theremin’s developments in the field of hidden recording of audio information, in other words, wiretapping, such as Buran and Zlatoust, you will probably be impressed by the experience of Israeli researchers described below. Ben Nassi, Aaron Pirutin, Yuval Elovici, Boris Zadov of Ben-Gurion University of the Negev, and Adi Shamir Shamir of the Weizmann Institute of Science has developed a device that can remotely listen to speech and other sounds from the vibrations of a light bulb hanging from the ceiling. The device decrypts data in real time and allows you to receive information almost instantly.

A bit about the history of photoacoustic wiretapping

The methods of wiretapping of this type are rooted in centuries of research by the engineer of the closed Tupolev design bureau and the pioneer of electronic music, Lev Theremin. Which back in the mid-forties of the last century developed the Buran system, it was able to listen to the vibration of window panes using reflected infrared rays. The same principle subsequently formed the basis of laser microphones. However, the method was not perfect. The presence of sound-absorbing barriers in front of the sound source prevented sufficient shaking of the glass in order to carry out any useful information pickup.





laser microphone of the late 80s



The advent of video cameras with high resolution and frame rate has opened up new possibilities for wiretapping. Sound waves, colliding with the surface of objects, cause vibrations imperceptible to the eye.







To recognize them, a high-resolution camera with a frame refresh rate of 60 fps or more can be used. Three years ago, a team of researchers at Massachusetts Institute of Technology was able to convert a video captured at 2200 fps into the sound of a melody that was played indoors at the time of shooting. Later it was found that the method can be applied with less efficiency even with a refresh rate of 60 fps.







This method also had limitations. Firstly, the cost of cameras with high and ultra-high refresh rates. Secondly, there are problems with the processing speed of an image shot at such a frame rate; voluminous video files require long processing, the duration of which directly depends on the hardware capacities. This limits the use of the realtime method.

Cameras with the existing resolution practically do not allow using the camera at a considerable distance, limiting it to 5-6 meters to the object.

The essence of the new method

Israeli scientists decided to improve the method of the Americans, focused the survey on a specific object with a telescope and replaced an expensive camera with an inexpensive photodiode. Air trembling during a conversation causes microvibrations of the light bulb, which in turn causes changes in illumination that are not noticeable, but significant for sensitive equipment. The light is captured by the telescope and converted by a photodiode into an electrical signal. Using a software analog-to-digital converter, the signal is recorded in the form of a spectrogram, which is processed by an algorithm written by the researchers and then converted into sound.



The researchers tested the efficiency of the method by laboratory experiment, in which they attached a gyroscope to a light bulb and reproduced sounds with a frequency of 100 to 400 Hz in one centimeter from the object. The oscillations of the bulb were small and ranged from 0.005 to 0.06 degrees (the deviation was on average from 300 to 950 microns), but the main thing was that they differed significantly depending on the frequency and sound pressure level, and, accordingly, there is a dependence of the oscillations from the characteristics of propagating sound waves.







The oscillations in the vertical and horizontal planes were very small (300–950 microns), but they varied depending on the frequency and volume of the delivered sound, which means that the light bulb, albeit barely noticeable, still vibrates from the sound waves propagating nearby, and fluctuations depend on their characteristics.

Measurements and experiment

Measurements of the data from the photodiode showed approximate changes in current when the light bulb was vibrated at different distances between it and the telescope. It was found that when using a 24-bit conversion, oscillations of a 300 micron light bulb in the plane cause a voltage change of 54 microvolts, which is quite enough to transmit the test spectrum (100 - 400 Hz) at a considerable (several tens of meters) distance using the optics of the used telescope. Also, the absence of sound is reflected in the spectrogram of the optical signal from the bulb in the form of a peak of 100 Hz (which is caused by its flicker frequency). This feature was also added to the algorithm.







The algorithm itself acts sequentially. At the first stage, it works as a filter of informationally insignificant frequencies, such as the flicker frequency, and then selects the spectrum corresponding to speech. After that, it removes the frequency signs of extraneous noise, similar to standard denoisers in voice recorders and studio recorders. The spectrogram processed in this way is converted into sound by a third-party program.







Lamphone, created by scientists, in the current version allows real-time reconstruction of speech and music from a room located 25 meters from the observation point. This is objectively proved by the following experiment, the installation, equipped with an amateur telescope with a 20-cm lens, was installed on a bridge, 25 meters from the window into the room where the lamp was located. Not far from the lamp, The Beatles' songs “Let It Be” and Coldplay “Clocks” were played, as well as a recording of a fragment of D. Trump's speech with the phrase “We will make America great again”.







As a result, the sound recordings reconstructed from the spectrograms turned out to be quite distinguishable, the melodies were easily guessed by the Shazam service, and the words were recognized by the open Google API for text recognition.

Dry residues

The device is working. Nothing like this has been reported before. This will somewhat simplify the work of the special services, and everyone who has something to fear should take new precautions. It is not yet clear whether the system can work with anything other than a moving light source. Israeli researchers plan to continue their research.