Huge power imbalance between firms and users whose info they grab

Pervasive data-gathering needs urgent action – report

Mass commercial data gathering and opaque decision-making processes have a “massive potential” to damage personal autonomy and dignity, a report has said.

Data gathered and stored by companies is increasingly being used to make decisions that can affect people’s lives, but the systems that drive these processes are often opaque, and frequently accused of bias.

And, according to an analysis of these networks of digital tracking and profiling, there is a “massive potential to limit personal agency, autonomy and human dignity” at both an individual and societal level.

The analysis, published this month (PDF), was carried out by Vienna-based digital rights researcher Wolfie Christl and extended a previous study of the networks companies have to assess how they affected people’s lives.

Christl’s work identified two dichotomies: the increasingly large divide between the power companies hold and the power of their customers or users; and the growing gap between large and small companies, as the bigger ones gain more data, and therefore more power.

“Through data-driven personalization, companies and other institutions can easily utilize information asymmetries in order to exploit personal weaknesses with calculated efficiency,” he wrote.

The abuse of personalisation and data-driven decision making results in three major challenges, Christl said.

The first is getting the distinction right between acceptable persuasive practices and beneficial personalisation on the one hand, and unacceptable manipulation and exploiting personal weaknesses on the other.

Second is addressing the power imbalance between companies and consumers, and the third is mitigating against the pervasive collection and trade of personal data when it happens without the data subjects even being aware of it.

In addition to these broad challenges, Christl set out seven more specific privacy issues that needed to be tackled urgently, which include targeting “disruptive” tech firms that operate in legal grey areas, and addressing the platform and data power of tech giants.

He also called for action on the “ubiquitous” sharing of personal data with third parties, such as advertising companies, and the use of data collected for one purpose being used for another.

Jim Killock, the executive director of the Open Rights Group – who described the report as “insightful and comprehensive” – said that it was important that conversations about data gathering looked beyond privacy.

“We should remember that this is going to have huge effects on things like competition between companies, the availability of products, and who is able to access services - as well as just thinking of this as a simple privacy issue,” he said.

The report comes as the question of data-driven decision making is under scrutiny in Parliament, through two select committee investigations into artificial intelligence and algorithms in decision-making ®.