AI is gradually moving into schools, promising personalised support, faster feedback and tools that can help teachers understand where pupils are struggling. But who owns a child’s data when AI enters the classroom, and how much information should technology companies be permitted to collect?
This question is becoming increasingly urgent as schools adopt systems capable of analysing far more than examination results. An AI education platform could process a child’s written work, learning speed, mistakes, behaviour, interests and communication style. Used over several years, it could build an extraordinarily detailed profile that follows that child throughout their education and possibly beyond it.
Microsoft Agrees to New School Privacy Rules
The issue received fresh attention this week after Microsoft announced a new AI safety and privacy agreement with the American Federation of Teachers and the United Federation of Teachers in the United States.
The agreement establishes protections that American school districts will be able to incorporate into their contracts with Microsoft. These include commitments that student and teacher data will not be used to train AI models, sold or repurposed.
Schools will retain control over how information is used, stored and deleted. Families should also receive clear explanations about what data is collected and how AI systems operate.
Crucially, AI will not be permitted to make important decisions about pupils without human oversight. The agreement also seeks to prevent harmful or manipulative interactions and prohibits AI companion chatbots within participating schools.
The Microsoft announcement describes these protections as legally enforceable rather than simply voluntary promises. School districts adopting the terms could potentially end agreements or seek damages if the rules were broken.
What Could Educational AI Learn About a Child?
Traditional school records already contain sensitive information, including academic results, attendance, disciplinary matters and details about additional educational needs. AI could take this considerably further.
A system used regularly might identify that a pupil reads more slowly than classmates, repeatedly struggles with particular concepts or becomes anxious when completing certain tasks. It might record patterns in their language, confidence, concentration and emotional responses that are not immediately obvious to a teacher.
Some of this could be helpful. AI might spot learning difficulties earlier, adapt lessons to suit individual needs or help teachers understand where additional support is required.
However, that information could also become a permanent digital judgement formed during childhood. A pupil who struggles at nine years old should not still be defined by the same AI profile at 16, 18 or even later in adult life.
Children develop at different rates. Their abilities, behaviour and confidence can change enormously. Yet data has a habit of surviving long after the circumstances that produced it have changed.
What Happens to Data About Autism and Special Needs?
The questions become even more serious when AI is used with children who have autism, anxiety, communication difficulties or other special educational needs.
AI might record meltdowns, sensory difficulties, social behaviour, emotional distress or problems communicating in class. It could analyse how frequently a child needs support, how they react to changes in routine or whether they appear able to work independently.
Who decides how long this information can be held? Does it remain with the school, the education authority or the company providing the AI system? Could it later influence decisions about qualifications, further education, employment or access to services?
There is also the question of whether the child will eventually have the right to see the information for themselves. A young adult could potentially request access to years of AI generated observations, predictions and assessments created when they were too young to understand that such a profile existed.
What would be the psychological effect of reading that an AI system had repeatedly classified them as anxious, disruptive, socially withdrawn or unlikely to achieve certain outcomes? Even if the assessment was never intended to cause harm, seeing yourself reduced to a collection of risks and predicted limitations could be deeply upsetting.
AI generated conclusions may also be wrong. Autism presents differently in every individual, while behaviour that appears unusual to a system may be a reasonable response to noise, anxiety, bullying or an unsuitable environment. Without proper context, AI could record the reaction while completely missing the cause.
Are Existing Data Protection Rules Enough?
Schools already have responsibilities to protect personal information, but AI introduces questions that existing policies may not answer clearly. It is no longer simply a matter of storing a child’s name, attendance record or examination results. AI can analyse behaviour, identify patterns and create conclusions that were never directly entered into the original record.
Parents need to know whether these AI generated observations are treated as part of the child’s official educational record, how long they can be retained and who has the right to challenge them. Effective data protection must cover not only the information originally collected, but also the new profiles and predictions that AI creates from it.
Should Parents Be Told More?
Many parents probably have little idea how much information educational technology can collect. Accepting a school’s technology policy is rarely the same as giving properly informed consent.
Parents should be told which AI systems are being used, what information they receive and whether any external company can access that data. They should know how long records are retained, where they are stored and whether they can request their correction or deletion.
The explanation must be written in ordinary language. Long privacy policies filled with legal and technical terminology may satisfy a formal requirement, but they do little to help families understand what is actually happening.
Schools should also consider whether parents and pupils can refuse certain uses of AI without placing the child at an educational disadvantage.
Not All AI Tools Are the Same
It would be a mistake to treat every use of AI in education as equally risky. A teacher using AI to suggest ideas for a lesson is very different from a system continuously monitoring and profiling individual pupils.
The greater the influence an AI system has over assessment, behaviour, support or future opportunities, the stronger the safeguards should be. An automated recommendation should never quietly become the deciding factor in whether a child receives additional help, enters a particular class or is considered capable of studying a subject.
Human oversight must mean more than a teacher simply approving whatever the system recommends. Educators need enough understanding of AI to question its conclusions and recognise when an answer may be incomplete, biased or wrong.
Children Should Not Pay for Innovation With Their Privacy
AI will almost certainly become a normal part of education. Used carefully, it could provide additional support, reduce teacher workloads and make learning more responsive to individual pupils.
But educational innovation should not create permanent records that follow children for years without their knowledge. Information collected to help a ten year old in the classroom should not quietly become a lifelong digital label.
The central principle should be simple. A child’s data belongs to the child. Schools may need to use it responsibly for education, but technology companies should never be free to collect, reuse or retain it simply because their software has entered the classroom.
Before AI becomes embedded throughout education, parents deserve clear answers about what is being recorded, how long it will survive and who might eventually be allowed to see it.
Learn more about AI, responsible adoption and the changing role of technology at aituitionhub.com.
Published: 10th September 2026.