The proposed EU regulation would set a global precedent for filtering the Internet, controlling who can access it, and taking away some of the few tools available for people to protect their right to private life in the digital space, Carmela Troncoso and Bart Preneel write.
ADVERTISEMENTThe European Parliament is currently debating a proposal for a regulation of the European Commission that intends to stop the online distribution of content depicting explicit activities involving a child and the online grooming of children.
First and foremost, child sexual abuse and exploitation are very serious crimes that can cause lifelong harm to the victims, while the online distribution of Child Sexual Abuse Material (CSAM) is growing.
It is the responsibility of government authorities, with the support of companies and communities, to undertake effective interventions which prevent these crimes and respond quickly when they occur.
Yet, as scientists, we have serious concerns about this proposal for several reasons: it has deep technical flaws that cannot be solved, and it is highly likely that the technology on which it relies will be abused.
Overall, the proposal will harm online privacy and security for all citizens (including children), and it presents serious problems in terms of human rights violations.
The technology is not there yet
The proposed regulation requires that service providers detect known CSAM content.
The problem is that the underlying technologies are not mature and highly intrusive; moreover, we do not see a realistic path to substantially improve them in the next decades.
The European Commission points to current approaches that are hard to deploy, are known to be easy to evade, and cannot be scaled to the volume of messages that would need to be scanned should this regulation pass.
In addition, the deployment of these technologies will lead to massive amounts of false positives — on the order of a million messages per day.
This means that many innocent citizens will be under investigation for serious crimes, which can be a traumatising experience.
AI-made errors could harm the innocent
In addition, the proposal imposes the detection of new CSAM images and videos and the detection of the grooming of children through text or audio with machine learning.
While AI has made enormous progress, these technologies are still far from perfect and will generate a large number of errors.
In particular, we have serious reservations about whether AI is capable of understanding the specific social context and subtle interactions.
Think of the father who lost access to his Google account because he sent pictures of his daughter to a medical professional or the teenager who is sending a photo to their boyfriend or girlfriend.
Raising alarms in these cases may have a chilling effect on the use of digital services.
The proposal could also make the internet less safe
The current proposal targets the detection of explicit material involving children. However, once such an infrastructure would be in place, function creep will likely set in: the same solution can be applied to detect terrorism recruitment, organised crime and less serious crimes.
Even more concerning is that less democratic governments will deploy it to automatically identify content critical of the regime.
The technologies proposed are by necessity not transparent and hide the content that is being searched for; this makes it much harder to detect function creep and abuse.
ADVERTISEMENT
A growing number of services has increased the protection of users by deploying end-to-end encryption.
This hampers scanning for CSAM content at the server side. The proposed regulation intends to bypass encryption technologies by scanning the content on the user's devices.
This kind of approach is similar to spyware: it creates a serious vulnerability that will facilitate other parties to gain access to that device.
In spite of the claims by the European Commission, there is no doubt that this approach will undermine encryption and make everyone’s communications less safe as a result.
Taking away people's right to a private digital life
Resources would be better spent on other approaches to protect children from sexual abuse.
ADVERTISEMENTAs an example, online services should make it much easier for users to complain about abuse — it is known that, in practice, complaints are an effective way to detect new abusive material.
In conclusion, the proposed regulation is bound to fail to achieve its goals.
In addition, it would set a global precedent for filtering the Internet, controlling who can access it, and taking away some of the few tools available for people to protect their right to private life in the digital space.
This will have a chilling effect on society and is likely to negatively affect democracies across the globe.
Carmela Troncoso is an Associate Professor at EPFL and head of the SPRING Lab, and Bart Preneel is a Full Professor at the University of Leuven and head of the COSIC group.
ADVERTISEMENTAt Euronews, we believe all views matter. Contact us at view@euronews.com to send pitches or submissions and be part of the conversation.