NIST digitally applied mask shapes to photos and tested the performance of face recognition algorithms developed before COVID appeared. Because real-world masks differ, the team came up with variants that included differences in shape, color and nose coverage. Credit: B. Hayes/NIST

Facial Recognition Programs Can’t See Who Those Masked People Are

Writer Stephen Johnson exposed glitches masks wreak on the facial recognition systems that watch us in a story on bigthink.com, where he asks the question: Do Americans even want better facial recognition technology?

The answer depends on who’s deploying the software. A 2019 survey from Pew Research Center found that 56 percent of Americans would trust law enforcement to use facial recognition technology responsibly, while 59 percent said it’s acceptable for officials to use the software to monitor public spaces for threats.

Americans are warier of trusting the private sector with facial recognition. For example, 36 percent of respondents said they’d trust technology companies to use the software responsibly, while only 16 percent said they’d trust advertisers to do the same.

Face masks shield wearers from not only viruses, but also facial recognition software, according to a new study from the U.S. National Institute of Standards and Technology (NIST).

The study tested how accurately, 89 commercial facial recognition algorithms could identify people who had masks on. To do that, the researchers tested a specific method of facial recognition called “one-to-one” matching, where the software compares a photo of a person to a different photo of the same person. The algorithm works by measuring the exact distance between a person’s facial features, and it’s the same technique used to unlock smartphones and verify passports.

The team tested the algorithms on about 6 million photos. Each photo set showed the same person twice: once with a digitally applied mask, once without. The results showed that masks effectively confused the software, causing a 5 to 50 percent spike in error rate in the algorithms.

With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces,” said Mei Ngan, a NIST computer scientist and an author of the report. “We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks. Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind.”

The researchers said they expect facial-recognition software will get better at recognizing people wearing masks.

No matter how Americans feel about facial recognition, it’s probably here to stay because of the investment and in-depth data available. The FBI already has a database of more than 641 million facial images, many of which simply come from publicly accessible social media posts. And even though cities like San Francisco have banned the technology, police across the country are using it with increasing frequency.

Still, as the mask catastrophe shows, there are limits to the effectiveness of facial recognition software.

read more at bigthink.com