Can reflections in eyeglasses actually leak info from Zoom calls? Here's a study into it

About time someone shone some light onto this

Boffins at the University of Michigan in the US and Zhejiang University in China want to highlight how bespectacled video conferencing participants are inadvertently revealing sensitive on-screen information via reflections in their eyeglasses.

With the COVID-19 pandemic and the rise in remote work, video conferencing has become commonplace. The researchers argue the ensuing privacy and security issues deserve further attention, and they've been casting an eye on this unusual attack vector.

In a paper distributed via ArXiv, titled, "Private Eye: On the Limits of Textual Screen Peeking via Eyeglass Reflections in Video Conferencing," researchers Yan Long, Chen Yan, Shilin Xiao, Shivan Prasad, Wenyuan Xu, and Kevin Fu describe how they analyzed optical emanations from video screens that have been reflected in the lenses of glasses.

"Our work explores and characterizes the viable threat models based on optical attacks using multiframe super resolution techniques on sequences of video frames," the computer scientists explain in their paper.

"Our models and experimental results in a controlled lab setting show it is possible to reconstruct and recognize with over 75 percent accuracy on-screen texts that have heights as small as 10 mm with a 720p webcam."

"The present-day 720p camera's attack capability often maps to font sizes of 50-60 pixels with average laptops," explained Yan Long, corresponding author and doctoral candidate at University of Michigan, Ann Arbor, in an email to The Register.

"Such font sizes can mostly be found in slide presentations and the headings/titles of some websites (for example, 'We saved you a seat in chat' on https://www.twitch.tv/p/en/about/)."

Being able to read reflected headline-size text isn't quite the privacy and security problem of being able to read smaller 9 to 12 pt fonts. But this technique is expected to provide access to smaller font sizes as high-resolution webcams become more common.

"We found future 4k cameras will be able to peek at most header texts on almost all websites and some text documents," said Long.

When the goal was to identify just the specific website visible on the screen of a video meeting participant from an eyeglass reflection, the success rate rose to 94 percent among the Alexa top 100 websites.

"We believe the possible applications of this attack range from causing discomforts in daily activities, e.g. bosses monitoring what their subordinates are browsing in a video work meeting, to business and trading scenarios where the reflections might leak key negotiation-related information," said Long.

He said the attack envisions both adversaries participating in conferencing sessions and also those who obtain and play back recorded meetings. "It would be interesting for future research to scrape online videos such as from YouTube and analyze how much information is leaked through glasses in the videos," he said.

A variety of factors can affect the legibility of text reflected in a video conference participant's glasses. These include reflectance based on the meeting participant's skin color, environmental light intensity, screen brightness, the contrast of the text with the webpage or application background, and the characteristics of eyeglass lenses. Consequently, not every glasses-wearing person will necessarily provide adversaries with reflected screen sharing.

With regard to potential mitigations, the boffins say that Zoom already provides a video filter in its Background and Effects settings menu that consists of reflection-blocking opaque cartoon glasses. Skype and Google Meet lack that defense.

The researchers argue other more usable software-based defenses involve targeted blurring of eyeglass lenses.

"Although none of the platforms supports it now, we have implemented a real-time eyeglass blurring prototype that can inject a modified video stream into the video conferencing software," they explain. "The prototype program locates the eyeglass area and applies a Gaussian filter to blur the area."

The Python code can be found on GitHub. ®