Facial-recognition algos vary wildly, US Congress told, as politicians try to come up with new laws on advanced tech

Most-accurate algorithms showed 'little to no bias', so nothing to fear, eh?

Vid A recent US government report investigating the accuracy of facial recognition systems across different demographic groups has sparked fresh questions on how the technology should be regulated.

The House Committee on Oversight and Reform held a hearing to discuss the dossier and surrounding issues on Wednesday. “Despite the private sector’s use of the technology, it’s just not ready for prime time,” said Rep Carolyn Maloney (D-NY), who chaired the meeting.

The report [PDF], published by America's National Institute of Standards (NIST) in December, reveals how accurate, or rather inaccurate, some of latest state-of-the-art commercial facial recognition algorithms really are.

NIST tested 189 algorithms submitted by 99 developers across a four datasets comprising of 18.27 million images taken of 8.49 million people.

“Contemporary face recognition algorithms exhibit demographic differentials of various magnitudes,” the report said. “Our main result is that false positive differentials are much larger than those related to false negatives and exist broadly, across many, but not all, algorithms tested. Across demographics, false positives rates often vary by factors of ten to beyond 100 times. False negatives tend to be more algorithm-specific.”

In other words, “different algorithms perform differently,” Charles Romine, director, of the Information Technology Laboratory at NIST and a witness at the hearing, explained. The rate of misidentifications in false positives and false negatives is dependent on the application.

The most risky applications were when false positives occurred in what Romine described as “one to many searches,” where an image is ran against a database of many images to look for a match. “False positives of one to many search is particularly important as the applications could include false accusations,” he said.

For example, a high risk one-to-many search would be matching people’s faces across a database of mugshots to look for suspected criminals. “This issue was not even on my radar until the ACLU study misidentified me,” said congressman Jimmy Gomez (D-CA), who was amongst one of the 28 politicians incorrectly identified amongst a database of mugshots in an experiment conducted by the American Civil Liberties Union.

“I have no doubt that it misidentified me because of my color. The technology is fundamentally flawed,” he added.

The NIST report highlighted the already well-established fact that most facial recognition systems struggle with identifying women, people of color, and the elderly, compared to white Caucasian men. False positives were between two and five times higher in women than men and were highest for West and East African and East Asian people, according to the investigation.

It should be noted, however, that the error rates for identifying East Asian people were less of a problem for facial recognition systems developed in East Asian countries like China, suggesting that the distribution of demographics in training data plays a big part in determining accuracy.

But for the most accurate algorithms identification problems across different demographics were diminished.

AI algorithms are rapidly improving over time

“Over the past year, I’ve seen headlines suggesting that facial recognition technology is inaccurate, inequitable, and invasive,” said Daniel Castro, a witness at the hearing and the veep and director of Center for Data Innovation at the non-profit Information Technology and Innovation Foundation. “If that was true then I would be worried too, but it isn’t.”

“There are many facial recognition systems on the market; some perform better than others across sex, gender, race, and age," he said. "Notably, the most accurate algorithms NIST has evaluated showed little to no bias. These systems continue to get measurably better every year and they can outperform the average human.”

Castro urged Congress to continue supporting computer vision research and to fund deployment federal use of facial recognition systems to improve security in federal buildings. He did, however, agree that application of the technology was particularly important and suggested that Congress consider enforcing legislation that would require law enforcement to obtain a warrant to track people’s movements if geolocation data is collected from facial recognition systems.


The Feds are building an America-wide face surveillance system – and we're going to court to prove it, says ACLU


Brenda Leong, the director of AI and ethics at the Future of Privacy Forum, also agreed that the context of how facial recognition was paramount. “The level of certainty acceptable for verifying an individual’s identity when unlocking a mobile device is below the standard that should be required for verifying that an individual is included in on a terrorist watch list,” she said.

If the most accurate algorithms are performing increasingly better across challenging demographics like gender, race, and age, will it be acceptable to use facial recognition systems in potentially riskier applications one day, the politicians wanted to know.

Meredith Whittaker, co-founder of AI Now, a research institute studying the social impacts of algorithms, said Congress should “halt the use of facial recognition in sensitive domains for private companies.”

Accuracy simply doesn’t seem to matter in some particularly concerning use cases, she opined. For example, Whittaker pointed out that algorithms used to analyse the facial expressions of candidates in job applications to look for certain characteristics are not backed by scientific evidence.

Instead, they could create a “bias feedback loop”, she explained, where the people that have already been awarded and promoted become a model for the type of people you want to hire. For example, if its white men in these higher positions then an AI could create a confirmation bias to prefer other white men.

Congress has been mulling over evidence in the hopes of crafting federal policies to regulate the technology for months. It held two other facial recognition hearings last year in May and July.

“We have a responsibility to not only encourage innovation, but to protect the privacy and safety of American consumers,” Maloney said. “Our committee is committed to introducing and marking up common sense facial recognition legislation in the very near future, and our hope is that we can do that in a truly bipartisan way.”

You can find a video replay of the hearing below. ®

Youtube Video

Broader topics

Other stories you might like

  • Demand for PC and smartphone chips drops 'like a rock' says CEO of China’s top chipmaker
    Markets outside China are doing better, but at home vendors have huge component stockpiles

    Demand for chips needed to make smartphones and PCs has dropped "like a rock" – but mostly in China, according to Zhao Haijun, the CEO of China's largest chipmaker Semiconductor Manufacturing International Corporation (SMIC).

    Speaking on the company's Q1 2022 earnings call last Friday, Zhao said smartphone makers currently have five months inventory to hand, so are working through that stockpile before ordering new product. Sales of PCs, consumer electronics and appliances are also in trouble, the CEO said, leaving some markets oversupplied with product for now. But unmet demand remains for silicon used for Wi-Fi 6, power conversion, green energy products, and analog-to-digital conversion.

    Zhao partly attributed sales slumps to the Ukraine war which has made the Russian market off limits to many vendors and effectively taken Ukraine's 44 million citizens out of the global market for non-essential purchases.

    Continue reading
  • Colocation consolidation: Analysts look at what's driving the feeding frenzy
    Sometimes a half-sized shipping container at the base of a cell tower is all you need

    Analysis Colocation facilities aren't just a place to drop a couple of servers anymore. Many are quickly becoming full-fledged infrastructure-as-a-service providers as they embrace new consumption-based models and place a stronger emphasis on networking and edge connectivity.

    But supporting the growing menagerie of value-added services takes a substantial footprint and an even larger customer base, a dynamic that's driven a wave of consolidation throughout the industry, analysts from Forrester Research and Gartner told The Register.

    "You can only provide those value-added services if you're big enough," Forrester research director Glenn O'Donnell said.

    Continue reading
  • D-Wave deploys first US-based Advantage quantum system
    For those that want to keep their data in the homeland

    Quantum computing outfit D-Wave Systems has announced availability of an Advantage quantum computer accessible via the cloud but physically located in the US, a key move for selling quantum services to American customers.

    D-Wave reported that the newly deployed system is the first of its Advantage line of quantum computers available via its Leap quantum cloud service that is physically located in the US, rather than operating out of D-Wave’s facilities in British Columbia.

    The new system is based at the University of Southern California, as part of the USC-Lockheed Martin Quantum Computing Center hosted at USC’s Information Sciences Institute, a factor that may encourage US organizations interested in evaluating quantum computing that are likely to want the assurance of accessing facilities based in the same country.

    Continue reading
  • Bosses using AI to hire candidates risk discriminating against disabled applicants
    US publishes technical guide to help organizations avoid violating Americans with Disabilities Act

    The Biden administration and Department of Justice have warned employers using AI software for recruitment purposes to take extra steps to support disabled job applicants or they risk violating the Americans with Disabilities Act (ADA).

    Under the ADA, employers must provide adequate accommodations to all qualified disabled job seekers so they can fairly take part in the application process. But the increasing rollout of machine learning algorithms by companies in their hiring processes opens new possibilities that can disadvantage candidates with disabilities. 

    The Equal Employment Opportunity Commission (EEOC) and the DoJ published a new document this week, providing technical guidance to ensure companies don't violate ADA when using AI technology for recruitment purposes.

    Continue reading
  • How ICE became a $2.8b domestic surveillance agency
    Your US tax dollars at work

    The US Immigration and Customs Enforcement (ICE) agency has spent about $2.8 billion over the past 14 years on a massive surveillance "dragnet" that uses big data and facial-recognition technology to secretly spy on most Americans, according to a report from Georgetown Law's Center on Privacy and Technology.

    The research took two years and included "hundreds" of Freedom of Information Act requests, along with reviews of ICE's contracting and procurement records. It details how ICE surveillance spending jumped from about $71 million annually in 2008 to about $388 million per year as of 2021. The network it has purchased with this $2.8 billion means that "ICE now operates as a domestic surveillance agency" and its methods cross "legal and ethical lines," the report concludes.

    ICE did not respond to The Register's request for comment.

    Continue reading
  • Fully automated AI networks less than 5 years away, reckons Juniper CEO
    You robot kids, get off my LAN

    AI will completely automate the network within five years, Juniper CEO Rami Rahim boasted during the company’s Global Summit this week.

    “I truly believe that just as there is this need today for a self-driving automobile, the future is around a self-driving network where humans literally have to do nothing,” he said. “It's probably weird for people to hear the CEO of a networking company say that… but that's exactly what we should be wishing for.”

    Rahim believes AI-driven automation is the latest phase in computer networking’s evolution, which began with the rise of TCP/IP and the internet, was accelerated by faster and more efficient silicon, and then made manageable by advances in software.

    Continue reading
  • Pictured: Sagittarius A*, the supermassive black hole at the center of the Milky Way
    We speak to scientists involved in historic first snap – and no, this isn't the M87*

    Astronomers have captured a clear image of the gigantic supermassive black hole at the center of our galaxy for the first time.

    Sagittarius A*, or Sgr A* for short, is 27,000 light-years from Earth. Scientists knew for a while there was a mysterious object in the constellation of Sagittarius emitting strong radio waves, though it wasn't really discovered until the 1970s. Although astronomers managed to characterize some of the object's properties, experts weren't quite sure what exactly they were looking at.

    Years later, in 2020, the Nobel Prize in physics was awarded to a pair of scientists, who mathematically proved the object must be a supermassive black hole. Now, their work has been experimentally verified in the form of the first-ever snap of Sgr A*, captured by more than 300 researchers working across 80 institutions in the Event Horizon Telescope Collaboration. 

    Continue reading
  • Shopping for malware: $260 gets you a password stealer. $90 for a crypto-miner...
    We take a look at low, low subscription prices – not that we want to give anyone any ideas

    A Tor-hidden website dubbed the Eternity Project is offering a toolkit of malware, including ransomware, worms, and – coming soon – distributed denial-of-service programs, at low prices.

    According to researchers at cyber-intelligence outfit Cyble, the Eternity site's operators also have a channel on Telegram, where they provide videos detailing features and functions of the Windows malware. Once bought, it's up to the buyer how victims' computers are infected; we'll leave that to your imagination.

    The Telegram channel has about 500 subscribers, Team Cyble documented this week. Once someone decides to purchase of one or more of Eternity's malware components, they have the option to customize the final binary executable for whatever crimes they want to commit.

    Continue reading

Biting the hand that feeds IT © 1998–2022