The Department of Homeland Security (DHS) is ditching a controversial data-mining program, the Associated Press revealed today.
The Analysis, Dissemination, Visualization, Insight and Semantic Enhancement program (ADVISE), a massive data-mining system under development at Lawrence Livermore Laboratory since 2003, was capable of analyzing one billion pieces per hour of "structured" information, such as databases, and one million pieces per hour of "unstructured" information, such as intelligence reports, emails or news articles.
The program had been quietly suspended in the spring due to privacy concerns, after reports surfaced that real personal data, rather than dummy information, had been used in the testing process. The Government Accountability Office (GAO) noted at the time that "the ADVISE tool could misidentify or erroneously associate an individual with undesirable activity such as fraud, crime or terrorism."
The program would have provided information to a slew of DHS agencies, including immigration, customs, border protection, biological defense and its intelligence office. The program had been temporarily halted to allow the DHS to develop stronger privacy protections for controversial program. DHS has apparently thrown in the towel on this one.
According to DHS spokesman Russ Knocke, "ADVISE is not expected to be restarted."
The report on the program completed in June by the DHS Office of Inspector General (OIG) seems to have been the nail in the coffin for this particular domestic intelligence operation. The report criticized the operation for failing to provide required privacy protections, as well as for failing to take into account the needs of the DHS "components" that ultimately would be expected to pay for the program. The DHS's own Office of Intelligence and Analysis (OIA) rejected the program in its entirety.
The DHS was cobbled together out of a variety of occasionally competing government agencies back in 2002, and the OIG report says as much about bureaucratic turf war and chaotic, failed attempts to run the government like a business as it does about the privacy implications of government data-mining. About $42m has been invested on a program that, through a combination of communication weakness and bureaucratic infighting, at times failed to obtain necessary information from the agencies or DHS "components" that were supposed to be the ultimate "customers".
The establishment of the DHS was intended to eliminate this kind of infighting, but old habits die hard. The appropriation of business terminology provides sadly comical insight into more general failings at the DHS, as its vast and ill-defined mandate led to generally confused interactions between its myriad agencies,"components" and leadership. The DHS Science and Technology Directorate (S&T), which had the reins on this project, never developed the required "business plan," and repeatedly failed to consult with "stakeholders" and "customers" - i.e., Immigration and Customs Enforcement (ICE), the OIA, and the Customs and Border Protection Office of Strategic Trade.
Most people, of course, just want their government to work, and don't concern themselves with the minutiae of Washington politics. Mass data-mining of the type that derailed the Total Information Awareness (TIA) initiative is more unnerving than petty infighting - the behavioral algorithms pioneered by credit rating agencies, based as they are on probabilities, lead to false positives on a regular basis and touch on constitutionally protected liberties.
The root of the dilemma is that effective data-mining requires the kind of mass data aggregation that ultimately and inevitably erodes human dignity and privacy.
Although the software program at the lab has been shelved, the Knocke noted that S&T had "determined that new commercial products now offer similar functionality while costing significantly less to maintain than ADVISE." Parts of the TIA continued, of course, under other names.
Mass government data-mining is most assuredly here to stay.®
Burke Hansen, attorney at large, heads a San Francisco law office