Mobile devices based eavesdropping of handwriting

Recent THaW paper:

When filling out privacy-related forms in public places such as hospitals or clinics, people usually are not aware that the sound of their handwriting can leak personal information. In this paper, we explore the possibility of eavesdropping on handwriting via nearby mobile devices based on audio signal processing and machine learning. By presenting a proof-of-concept system, WritingHacker, we show the usage of mobile devices to collect the sound of victims’ handwriting, and to extract handwriting-specific features for machine learning based analysis. An attacker can keep a mobile device, such as a common smartphone, touching the desk used by the victim to record the audio signals of handwriting. Then, the system can provide a word-level estimate for the content of the handwriting. Moreover, if the relative position between the device and the handwriting is known, a hand motion tracking method can be further applied to enhance the system’s performance. Our prototype system’s experimental results show that the accuracy of word recognition reaches around 70 – 80 percent under certain conditions, which reveals the danger of privacy leakage through the sound of handwriting.

July 2020: Tuo Yu, Haiming Jin, and Klara Nahrstedt. Mobile devices based eavesdropping of handwritingIEEE Transactions on Mobile Computing 19(7), pages 1649–1663, July 2020. IEEE. DOI: 10.1109/TMC.2019.2912747 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s