Police responding to ShotSpotter's AI-generated alerts of gunfire find evidence of actual gun-related crime only about one time in ten, a Chicago public watchdog has found.
The California biz uses machine-learning algorithms to determine whether loud bangs caught by microphones deployed across more than 100 US cities are gunshots or not. If a shot is identified, the location of the noise is triangulated and sent to the police as an immediate, real-time alert, and reports are later compiled for prosecutors for use in court cases.
ShotSpotter is under the microscope right now because a 65-year-old man spent almost a year behind bars awaiting trial for murder – and the primary evidence against him was a disputed ShotSpotter report of a gunshot.
Michael Williams was last year accused of shooting and killing 25-year-old Safarian Herring, and denied any wrongdoing. Prosecutors said ShotSpotter picked up the sound of gunfire right where and when Williams was seen in his car, in Chicago, giving Herring a ride. Williams said Herring was hit in a drive-by shooting.
Crucially, Williams' lawyers asked the trial judge to probe the ShotSpotter evidence after it emerged the AI software actually picked up a firework a mile away and this information was later revised by ShotSpotter staff. In response, the prosecution withdrew the ShotSpotter report, and last month asked for the case to be dismissed as it no longer had sufficient evidence. The judge agreed, and Williams was released as a free man.
Don't worry, someone did a probe
The City of Chicago's Office of Inspector General (OIG) decided to dig into the alerts ShotSpotter sends to the Chicago Police Dept (CPD) and the city's Office of Emergency Management. The city had a $33m three-year contract with ShotSpotter that was due to run out in August 2021, and in December last year, that contract was extended ahead of its expiry to mid-2023.
The watchdog said 50,176 alerts generated by ShotSpotter in Chicago between January 2020 and May this year probably were the result of gunfire, and were assigned unique IDs and had officers show up. Of those alerts, 41,830 resulted in some kind of police action, known as a disposition. And of those dispositions, only 4,556 indicated that "evidence of a gun-related criminal offense was found," the auditors said.
Thus, only 9.1 per cent of ShotSpotter alerts led to the police finding evidence of an actual gun crime. This did not go down well with the auditors, given that the ShotSpotter contract has been renewed.
“The CPD data examined by OIG does not support a conclusion that ShotSpotter is an effective tool in developing evidence of gun-related crime,” the watchdog said this week.
- A man spent a year in jail on a murder charge that hinged on disputed AI evidence. Now the case has been dropped
- British Army develops AI shotgun drone with machine vision for indoor use
- Apple's bright idea for CSAM scanning could start 'persecution on a global basis' – 90+ civil rights groups
- Cop drone crashes into flight instructor's airplane
The police, meanwhile, defended their use of the service.
“In order to reduce gun violence, knowing where it occurs is crucial,” a Chicago police spokesperson told The Register. “ShotSpotter has detected hundreds of shootings that would have otherwise gone unreported. ShotSpotter is among a host of tools used by the CPD to keep the public safe and ultimately save lives.
ShotSpotter has detected hundreds of shootings that would have otherwise gone unreported
“Using ShotSpotter, CPD receives real-time alerts of detected gunfire enabling patrol officers to arrive at a precise location of a shooting event quickly. Instead of relying on the historically low rate of 911 calls, law enforcement can respond more quickly to locate and aid victims, identify witnesses, and collect forensic evidence.
"The system gives police the opportunity to reassure communities that law enforcement is there to serve and protect them and helps to build bridges with residents who wish to remain anonymous.”
Chicago’s OIG isn’t convinced. In a 30-page report [PDF], the watchdog said the technology may lead to over-policing by sending officers into communities in search of serious crimes that never happened.
“Our study of ShotSpotter data is not about technological accuracy, it’s about operational value,” Deputy Inspector General for Public Safety Deborah Witzburg stressed.
“If the Department is to continue to invest in technology which sends CPD members into potentially dangerous situations with little information – and about which there are important community concerns – it should be able to demonstrate the benefit of its use in combatting violent crime.
This technology is changing the way CPD members interact with members of Chicago’s communities
“The data we analyzed plainly doesn’t do that. Meanwhile, the very presence of this technology is changing the way CPD members interact with members of Chicago’s communities.”
ShotSpotter’s software and hardware is proprietary, and hasn’t been openly audited for its accuracy. The company says its algorithms are 97 per cent accurate. “It is important to point out that the CPD continually describes ShotSpotter as an important part of their operations,” a ShotSpotter spokesperson told The Register.
“The OIG report does not negatively reflect on ShotSpotter’s accuracy which has been independently audited at 97 percent based on feedback from more than 120 customers. Nor does the OIG propose that ShotSpotter alerts are not indicative of actual gunfire whether or not physical evidence is recovered.”
The Register has asked Chicago’s OIG for further comment. ®