Cops responding to ShotSpotter's AI alerts rarely find evidence of gun crime, says Chicago watchdog
It may hurt community policing, too
Police responding to ShotSpotter's AI-generated alerts of gunfire find evidence of actual gun-related crime only about one time in ten, a Chicago public watchdog has found.
The California biz uses machine-learning algorithms to determine whether loud bangs caught by microphones deployed across more than 100 US cities are gunshots or not. If a shot is identified, the location of the noise is triangulated and sent to the police as an immediate, real-time alert, and reports are later compiled for prosecutors for use in court cases.
ShotSpotter is under the microscope right now because a 65-year-old man spent almost a year behind bars awaiting trial for murder – and the evidence against him involved a ShotSpotter report of a gunshot.
Michael Williams was last year accused of shooting and killing 25-year-old Safarian Herring, and denied any wrongdoing. Prosecutors said ShotSpotter picked up the sound of gunfire right where and when Williams was seen in his car, in Chicago, giving Herring a ride. Williams said Herring was hit in a drive-by shooting.
Williams' lawyers asked the trial judge to probe the ShotSpotter evidence, alleging in a court filing that ShotSpotter picked up the sound of a firecracker a mile away from where police said Herring was shot and that these details were later corrected by ShotSpotter employees to be gunfire where officers said Herring was hit.
Editor's note: ShotSpotter has responded to the allegations raised by Williams' lawyers, stating that, for its court evidence, its algorithm identified two data points: the exact coordinates where Herring was shot at the junction of South Stony Island Avenue and East 63rd Street, and the street address to the entrance of Jackson Park, the edge of which is where Herring was hit. The park's entrance is a mile from where the shooting occurred. These data points were not changed at any time.
ShotSpotter also said the reclassification of the sound from a firecracker to gunfire was innocuous: a human reviewer checked the audio and changed it from a possible firework to gunshot within a minute of its detection.
Thus, though Williams' lawyers sought to paint ShotSpotter's location and classification as ambiguous and unreliable, it is clear from the evidence why two data points were given – the precise coordinates of the actual shot; and what the algorithm thought was the nearest relevant street address, the adjacent park – and that these data points were not changed by ShotSpotter staff, and also how the sound was reclassified immediately by an employee. We are happy to clarify this situation.
Following the defense's request for an inquiry, the prosecution withdrew the ShotSpotter report, and last month asked for the case to be dismissed as it no longer had sufficient evidence. The judge agreed, and Williams was released as a free man.
The City of Chicago's Office of Inspector General (OIG) decided to dig into the alerts ShotSpotter sends to the Chicago Police Dept (CPD) and the city's Office of Emergency Management. The city had a $33m three-year contract with ShotSpotter that was due to run out in August 2021, and in December last year, that contract was extended ahead of its expiry to mid-2023.
The watchdog said 50,176 alerts generated by ShotSpotter in Chicago between January 2020 and May this year probably were the result of gunfire, and were assigned unique IDs and had officers show up. Of those alerts, 41,830 resulted in some kind of police action, known as a disposition. And of those dispositions, only 4,556 indicated that "evidence of a gun-related criminal offense was found," the auditors said.
Thus, only 9.1 per cent of ShotSpotter alerts led to the police finding evidence of an actual gun crime. This did not go down well with the auditors, given that the ShotSpotter contract has been renewed.
“The CPD data examined by OIG does not support a conclusion that ShotSpotter is an effective tool in developing evidence of gun-related crime,” the watchdog said this week.
- A man spent a year in jail on a murder charge that hinged on disputed AI evidence. Now the case has been dropped
- British Army develops AI shotgun drone with machine vision for indoor use
- Apple's bright idea for CSAM scanning could start 'persecution on a global basis' – 90+ civil rights groups
- Cop drone crashes into flight instructor's airplane
The police, meanwhile, defended their use of the service.
“In order to reduce gun violence, knowing where it occurs is crucial,” a Chicago police spokesperson told The Register. “ShotSpotter has detected hundreds of shootings that would have otherwise gone unreported. ShotSpotter is among a host of tools used by the CPD to keep the public safe and ultimately save lives.
ShotSpotter has detected hundreds of shootings that would have otherwise gone unreported
“Using ShotSpotter, CPD receives real-time alerts of detected gunfire enabling patrol officers to arrive at a precise location of a shooting event quickly. Instead of relying on the historically low rate of 911 calls, law enforcement can respond more quickly to locate and aid victims, identify witnesses, and collect forensic evidence.
"The system gives police the opportunity to reassure communities that law enforcement is there to serve and protect them and helps to build bridges with residents who wish to remain anonymous.”
Chicago’s OIG isn’t convinced. In a 30-page report [PDF], the watchdog said the technology may lead to over-policing by sending officers into communities in search of serious crimes that never happened.
“Our study of ShotSpotter data is not about technological accuracy, it’s about operational value,” Deputy Inspector General for Public Safety Deborah Witzburg stressed.
“If the Department is to continue to invest in technology which sends CPD members into potentially dangerous situations with little information – and about which there are important community concerns – it should be able to demonstrate the benefit of its use in combatting violent crime.
This technology is changing the way CPD members interact with members of Chicago’s communities
“The data we analyzed plainly doesn’t do that. Meanwhile, the very presence of this technology is changing the way CPD members interact with members of Chicago’s communities.”
ShotSpotter’s software and hardware is proprietary, and hasn’t been openly audited for its accuracy. The company says its algorithms are 97 per cent accurate. “It is important to point out that the CPD continually describes ShotSpotter as an important part of their operations,” a ShotSpotter spokesperson told The Register.
“The OIG report does not negatively reflect on ShotSpotter’s accuracy which has been independently audited at 97 percent based on feedback from more than 120 customers. Nor does the OIG propose that ShotSpotter alerts are not indicative of actual gunfire whether or not physical evidence is recovered.”
The Register has asked Chicago’s OIG for further comment. ®