Data Privacy

AI_in_the_Classroom__Protecting_Student_Data (1).mp4

Definition

Data Privacy relates to the collection and protection of personl information, in addition to the normal data, this also includes learning behaviours, disabilities, socio-econonmical factors. Schools and educational organisations will have detailed policies, procedures and guidelines around the management of this data.

The Department of Educuation (2023) sets out the following in regards to the management of Privacy, Security and Safety regarding Data Privacy:

Privacy and data protection: generative AI tools are used in ways that respect and uphold privacy and data rights, comply with Australian law, and avoid the unnecessary collection, limit the retention, prevent further distribution, and prohibit the sale of student data.

Privacy disclosure: school communities are proactively informed about how and what data will be collected, used, and shared while using generative AI tools, and consent is sought where needed.

Protection of student inputs: students, teachers and staff take appropriate care when entering information into generative AI tools which may compromise any individual’s data privacy.

Cyber-security and resilience: robust cyber-security measures are implemented to protect the integrity and availability of school infrastructure, generative AI tools, and associated data.

Potential Benefits

Following careful selection of AI tools that meet due diligence requirements for data and privacy, can promote equity and inclusion
Prior to and whilst using AI students and teachers can hold conversations on consent regarding data collection and use (esafety Commissioner, 2022)

Limitations and Risks

Personal information of students can be embedded by off-site commercial organisations and AI developers (Franks, 2024)
Student data may be input which may compromise individual rights to privacy (Franks, 2024)
In appropriate use or disclosure of information may result in human harm (eSafety Commissioner, 2022)
turning students into datapoints

Real World Example 1

A Year 9 maths teacher provides a differentiated learning activity to a student through a link to a new digital tool that includes AI tutoring. Despite receiving digital safety training at the school, as the student usually requires one-on-one support in class, they autonomously click on the link, sign up and as a result disclose personal information including their school name, school email and date of birth.

In this instance, the teacher may not have followed the appropriate due diligence process exposing the student and potentially the school to a data breach.

(eSafety Commissioner, 2022)

Real World Example 2

A school introduces an AI-powered tutoring platform to support personalised learning. The tool tracks student interactions—such as response times, keystrokes, and even emotional cues via webcam—to adapt content delivery. While the intention is to enhance engagement, the platform also collects and stores sensitive data including behavioural patterns, academic performance, and biometric indicators.