Online influences

Although AI tools carry benefits and can be used positively in daily life, they can also be used for harmful and illegal purposes, such as the creation of sexual imagery of children. 

What is AI generated child sexual abuse material?

Artificially generated child sexual abuse material describes images of child sexual abuse that are partially or entirely computer-generated. They are usually produced using software which converts a text description into an image. This technology is developing rapidly, the images created can now be very realistic, and recent examples are difficult to differentiate from unaltered photographs.

Many popular, publicly available artificial intelligence tools automatically block attempts to create abusive material, but the large number of child sexual abuse images made using them that have been detected show that individuals have found ways around this.

Some artificially generated child sexual abuse material depicts children who appear to be entirely artificial or fictional, and it is often assumed this means no child has been harmed or affected. But most tools rely on thousands of existing images to inform or ‘train’ them, so genuine images of individuals are likely to have been used as reference material. 

There are tools and methods that allow users to deliberately make images or videos that feature the likenesses of real people. This can allow someone to create sexual abuse images and videos with children and young people as the subjects without being present or having their knowledge or consent. Examples of this include the use of ‘deepfake’ tools, which can edit a child into existing explicit or abusive material, or apps which digitally remove the clothes of subjects in photographs.

Legality

It is llegal to create, view and share all sexual images of children under-18 produced using artificial intelligence. It is important to remember this applies to all material depicting child sexual abuse – it doesn’t matter if the material is created in a ‘conventional’ way using a camera, or created using artificial intelligence tools.

How to respond

The skills professionals should use to respond are the same as in any case of child sexual abuse. Although you may not be certain about how images have been created, or worry that you don’t have specialist knowledge in responding to harm in this context, this shouldn’t change the overall protection and support response that should be followed for any concern about child sexual abuse.

Responding to children captured in AI generated CSA material

Your response may be different depending on the individual circumstances a child is in. But in all cases, you should refer concerns to children’s social care and/or the police if you’re concerned that a child or young person has been or is at risk of immediate harm at any point.

The impact on victims of the circulation of images of sexual abuse of children can be as severe and varied as the abuse itself. In cases where the images are partially or entirely artificially generated, it’s important for professionals to be there to listen to children and young people, ask them what happened, believe them and tell them how you can help.

Support for children and young people should always respond to their own individual situation and needs, which you can do regardless of the technology used to harm them.

Responding to adults who have created, viewed or shared AI generated CSA material

Professionals should build their understanding of who has accessed this material and why, the risk they may pose to the children around them, and consider what safety plans should be put in place to keep everyone safe from harm.

Even where you suspect or an adult has told you the images were created using artificial intelligence, it is still important to undertake an assessment of risk and, if this has not happened already, report them to the police

See the Centre of Expertise's guide on managing risk and trauma after online sexual offending for more information: https://www.csacentre.org.uk/research-resources/practice-resources/managing-risk-and-trauma-after-online-sexual-offending/ 

Responding to children who may have viewed or created artificially generated child sexual abuse material

Apply the same considerations as you would in other cases of harmful sexual behaviour. 

First, refer them to children’s social care and/or the police if you’re concerned that a child has been or is at risk of immediate harm, and consider the impact on each child involved.

Many of the young people you will work with in these contexts will, with the right intervention at the right time, be unlikely to present a risk as adults.  You may also meet some young people whose behaviour is particularly worrying, which may indicate they have experienced harm or abuse of some form themselves. Their needs must be addressed in your professional response too, including where relevant with the family.