As generative AI becomes increasingly widespread and accessible, the use of AI in schools, and by students at home, has become the subject of growing debate. However, everyone agrees that AI is here to stay and should therefore form part of a school’s wider filtering and monitoring considerations.
Supporting KCSIE expectations
Statutory KCSiE 2026 guidance now includes AI-enabled risks as part of safeguarding in education and expects school and college settings to integrate AI into a Whole School Approach for online and digital safety.
A robust monitoring solution must account for AI tools in use, both online and offline, with effective checks and recorded findings taking place every academic year.
A core safety requirement within KCSiE includes AI generated content, deepfakes, AI driven harassment, misinformation and manipulation through blackmail risks using imagery, whilst also observing early identification of AI-assisted bullying and the impact on wellbeing. School policies should reflect these changes!
Some of the safeguarding concerns this raises for schools include:
- Bullying and harassment. AI can be used to create abusive, humiliating and targeted content. Securus can identify concerning language and provide contextual activity and screenshots for review.
- Attempts to bypass filtering. Pupils may use/ask AI how to circumvent filtering. Monitoring helps identify these attempts even when different terminology or tools are used.
- Antisemitism, racism, hate and discrimination. AI can be prompted to generate antisemitic, racist or discriminatory and hateful material. Securus monitors relevant terms and phrases within a dedicated safeguarding category for such topics.
- Over-reliance on AI. Pupils may use AI for emotional support or advice, often treating it as a source of truth.
- AI-generated sexual or abusive content. Generative AI can be misused to create sexualised, degrading or fabricated content. Monitoring can help identify concerning activity at an early stage.
- Self-harm and suicide. A pupil might ask AI questions relating to self-harm, suicide or concealing concerning behaviour. Securus captures can provide an early-warning safeguarding indicator.
- Grooming and manipulation. AI can help create messages, false personas and personalised communications that could be used for grooming, coercion and/or manipulation.
- AI-assisted cyber attack/misuse. Pupils could ask AI to generate phishing messages or methods of accessing accounts. Context provided by Securus helps distinguish legitimate computing activity from potentially harmful behaviour.
- Using AI to cheat. Securus has previously covered this issue in-depth, which you can read here.
Securus inspects AI-generated content for safeguarding concerns
Securus also uses AI within the monitoring process to help distinguish potentially concerning captures from neutral activity, reducing unnecessary noise for safeguarding teams.
AI training data is used to better identify trends and patterns in behaviour, and to ensure the core library is current and relevant.
Book a demo here with the Securus safeguarding team, to learn more about our digital monitoring solutions for education and our response to the increasing use of AI in education settings.