Your computer mouse has big ears.

Image courtesy of GPT4/Dall-E-3, generated using the keywords "computer mouse with big ears and a microphone as a scroll wheel."

The accessibility of these advanced input devices is steadily increasing. Consumer-grade mice with high-fidelity sensors are already readily available for under 50 U.S. Dollars. As improvements in process technology and sensor development continue, it is reasonable to expect further decreases in price, similar to the trend shown in the picture above. Ultimately, these developments entail an increased usage of vulnerable mice by consumers, companies, and governmental entities, expanding the attack surface of vulnerabilities in these advanced sensor technologies. 

With only a vulnerable mouse, and a victim’s computer running compromised or even benign software (in the case of a web-based attack surface), we show that it is possible to collect mouse packet data and extract audio waveforms. Moreover, the software used scheme used in our pipeline is invisible to the average user during the data collection process. After this stage, all signal processing and data analysis can be performed offsite at any time the adversary wishes. We present a visual outline of our pipeline in the above diagram.

Our target for a suitable exploit delivery vehicle is open-source applications where the collection and distribution of high-frequency mouse data is not inherently suspicious. Therefore, creative software, video games, and other high performance, low latency software are an ideal targets for injecting our exploit. Furthermore, we note that extracting the collected mouse data from the victim's computer is a task that requires ample consideration. Many video games often contain networking code that can be reused by our exploit without raising suspicion. Thus, using a video game as the delivery vehicle of our exploit allows us to meet the performance demands of our collection scheme. 

A hypothetical timeline of an attacker using the Mic-E-Mouse attack is presented in the figure above.

Our pipeline overcomes significant roadblocks – including heavy quantization, non-uniform sampling, and a high noise floor – in order to obtain a comprehensible output signal. Ultimately, we show that auditory surveillance through high-performance optical sensors is now possible, effective, and performant.

The representation of phoneme frequency versus perceived loudness presented in the above diagram is also known as a Speech Banana by otolaryngologists and speech pathologists. From this diagram, it is evident that the majority of human speech falls between the frequency range of 200Hz to 2000Hz, and is therefore detectable by our pipeline. We overlay an audio waveform from the filtering stage of the Mic-E-Mouse pipeline to show the correlation between phoneme frequency and signal strength. Further context is provided in the full paper.

We provide a demonstration of the Mic-E-Mouse pipeline in the above video.

Anonymous repository access has been provided to reviewers during the review process using the following link:

https://anonymous.4open.science/r/Mic-E-Mouse-7028/README.md

Anonymous Data access has been provided to reviewers during the review process using the following link:

 https://drive.google.com/drive/folders/1DcTldouupfp7BMteE1Br0lq7RCdQQ0Hc?usp=drive_link