AI-created child porn increases like an avalanche – Sydsvenskan

Child pornography produced with the help of AI tools is increasing at a rapid rate according to the children’s rights organization Ecpat, reports P4 Skaraborg in Sveriges Radio.

– It has gone from almost zero in December to that we see it every day, says Thomas Andersson, responsible for Ecpat’s tip line.

– You can take a picture of, for example, a neighbor’s child and use it as a basis for the production. When this is spread, it is easy to understand the damage it entails, says Thomas Andersson at the children’s rights organization Ecpat. Archive image.

Photo: Naina Helén Jåma/TT

With ever-improving AI tools, it is becoming more difficult for Ecpat’s analysts to determine whether abuse material is computer generated, or from a real abuse. It may even be that AI-generated child pornography is already being mistaken for real video.

– If it hasn’t already happened, it’s only a matter of time, says Thomas Andersson to TT.

Violates guidelines

In discussion threads on the shady darknet, Ecpat has been able to see how people collaborate to find ways to trick AI systems into breaking their guidelines and creating child pornography. The program’s restrictions don’t work, according to Andersson.

– They also collaborate to train the tools to create more realistic images.

On one of the forums, 80 percent of users wrote that they already use AI to generate abuse material, or that they want to learn.

There are several problems with the development. Among other things, it will be more difficult for Ecpat and the police to detect ongoing abuse when more resources are required to filter out material created with AI.

Requirements for companies

Ecpat has also already been able to see examples of vulnerable children being exploited again when their images are used as the basis for new, AI-generated, material.

– It will be an abuse that will not end.

One way to access AI-generated child pornography is to place greater demands on the platforms that store images and videos online. They need mandatory tools to detect, report and remove abusive material, regardless of whether it is photographed, filmed or AI-generated, says Thomas Andersson.

– It also needs to be stopped in the production stage, and there it depends a lot on those who manufacture the AI tools.