What Schools Get Wrong About AI (And What Actually Matters)

By Rico Tan — IT Director & AI Governance Consultant

I sat in a conference room last fall with an elementary school’s leadership team. They had just rolled out an AI writing assistant to every student in grades 4 through 8. No policy. No teacher training. No conversation with parents. The superintendent was beaming. “We’re ahead of the curve,” he said.

Three weeks later, a parent called to report that her ten-year-old had used the tool to generate a book report — and the AI had included a paragraph hallucinating a fictional author as real. The child submitted it. The teacher didn’t catch it. It ended up in a district-wide literary showcase.

That’s not a technology problem. That’s a leadership problem.

I’ve spent the last two years working directly with K-8 schools and nonprofits on AI governance — not the glamorous strategy consulting version, the real version. The one where you’re reading the terms of service for a tool a teacher found on TikTok at 11pm. I’ve seen what goes wrong when institutions move fast on AI without a framework. And I’ve seen what works.

Here’s what schools keep getting wrong.

Wrong Assumption #1: “We Need a Policy First”

This one sounds responsible. It isn’t.

Every school I work with eventually produces a version of the same AI policy document: a two-page PDF that defines what AI is, prohibits “academic dishonesty,” and promises a review “at the end of the academic year.” It takes three months to write. It sits in a shared Google Drive folder. Nobody reads it.

The problem isn’t that schools write policies. The problem is that they treat the policy as the destination instead of the starting line. A document doesn’t change behavior. Training does. Conversation does. Repeated modeling of expectations does.

One district IT director I work with put it well: “We spent six weeks on the policy and six hours on implementation. Then we wondered why nothing changed.”

A policy without operational support is just liability documentation. If you have a policy but haven’t run a single professional development session on what AI-assisted work actually looks like, you’re not ahead of the curve. You’re just covered on paper.

Wrong Assumption #2: “Teachers Will Figure It Out”

Teachers are resourceful. That’s the problem.

When you drop an AI tool into a school without guidance, the most resourceful teachers adopt it fastest — often in ways that were never intended. I’ve seen a well-meaning educator use a general-purpose AI chatbot to generate individualized math feedback for 28 students, not realizing she was pasting student names and performance data into a tool with no data processing agreement. FERPA implications aside, the outputs were also wrong for six of the students.

This isn’t a teacher failure. She was trying to do her job better with the tools available. It’s a system failure — the school gave her a tool, assumed competence, and provided zero guardrails.

Teachers need to know: what data can go in, what can’t, what to do when the AI is wrong, and what “good enough to use” looks like for their specific context. That’s not a one-size-fits-all training module. That’s a conversation that happens at the department level, repeatedly, across the year.

Wrong Assumption #3: “Students Are Already Using It — So We Should Too”

Yes, students are using AI. That’s not a reason to accelerate adoption without intention — it’s a reason to teach critical evaluation.

The reactive version of this logic sounds like: “They’re using it anyway, so we might as well bring it into the classroom and make it official.” The problem is “official” without “intentional” just normalizes the behavior without building the skill.

What students actually need isn’t more AI exposure. They’re getting plenty. What they need is structured practice evaluating AI outputs — checking claims, recognizing hallucinations, understanding when AI assistance improves their thinking and when it replaces it. That’s the competency gap. And it’s a curriculum design challenge, not a technology procurement challenge.

One K-8 school I consulted with resisted the urge to “go all in” on AI tools and instead built a six-week media literacy unit that included AI output evaluation alongside news literacy and source verification. End-of-unit assessments showed measurable improvement in students’ ability to identify false or misleading AI-generated content. That’s the win. Not the tool count.

Wrong Assumption #4: “The Vendor’s Privacy Policy Covers Us”

It does not. Or at least: not in the way you think.

I have read hundreds of EdTech privacy policies. Most of them are written to protect the vendor, not the school. The language around data retention, training data use, third-party sharing, and student data definitions is almost always ambiguous — and ambiguity in a privacy policy is not a protection. It’s a gap the vendor can drive a truck through.

Schools have FERPA obligations. In California, there’s also SOPIPA and AB 1584. These are not abstract compliance checkboxes. They mean that before any AI tool touches student data, your school needs a signed data processing agreement that explicitly covers: what data is collected, how long it’s retained, whether it’s used to train models, and how it’s deleted upon request.

Most free AI tools used in classrooms — the ones teachers find and adopt without going through IT — have none of this in place. “Free” in EdTech almost always means “the data is the product.” That’s not an accusation. It’s a business model. Know what you’re buying before you call it a win.

Wrong Assumption #5: “AI Will Save Us Time”

Sometimes. With the right tools, the right training, and the right expectations — yes, AI can genuinely reduce administrative burden and help teachers focus on what matters. I’ve seen it work.

I’ve also seen a principal spend four hours per week reviewing AI-generated lesson plan drafts because the outputs kept missing the school’s curriculum standards. She spent more time correcting the AI than she would have spent writing the plans herself. But the tool was in the budget, the vendor had a great pitch deck, and stopping would have felt like failure.

AI saves time when the task is well-defined, the inputs are clean, the outputs are verifiable, and the user has enough domain expertise to spot errors quickly. That’s a narrow set of conditions. Most of the time-saving claims you’ll hear at EdTech conferences are based on controlled pilots with enthusiastic early adopters — not on the messy reality of a 400-student school with three IT staff and no dedicated AI coordinator.

Before you promise time savings, run a 30-day pilot on one specific use case. Measure it honestly. That’s the data you actually need.

Wrong Assumption #6: “This Is an IT Problem”

I am an IT director. I am telling you: this is not an IT problem.

IT can evaluate tools for security and compliance. IT can set up access controls and manage vendor agreements. IT can flag data risks and enforce acceptable use policies. But IT cannot tell you what pedagogical role AI should play in a third-grade writing class. That’s a curriculum decision. IT cannot tell you how to have the conversation with a parent whose child doesn’t understand where their homework ends and the AI begins. That’s a communication and culture decision.

AI governance in schools is inherently cross-functional. It requires instructional leadership, legal/compliance, communications, and IT — working together from the start. When it gets handed entirely to IT, you end up with technically secure tools that nobody knows how to use well. When it gets handed entirely to curriculum staff, you end up with pedagogically interesting tools that are a privacy nightmare.

The schools doing this well have a working group. Not a committee that meets once a semester. A working group that meets monthly, reviews real incidents, iterates on guidance, and has actual authority to pause tool deployments when something goes wrong.

What Actually Matters

After two years of this work, here’s my honest distillation:

• Start with use cases, not tools. Pick two or three specific problems you want to solve. Find tools that address those problems. Evaluate them rigorously. Resist the urge to buy the platform that promises to do everything.
• Protect student data as a non-negotiable. If a tool doesn’t have a signed DPA, it doesn’t get used with students. Period. There is no exception for “it’s really useful.”
• Train on judgment, not just mechanics. Teachers don’t need to know how a large language model works. They need to know what good AI-assisted output looks like, what bad output looks like, and what to do when they’re not sure.
• Build feedback loops. Create a simple way for teachers to report when an AI tool produced something wrong, harmful, or inappropriate. Review those reports monthly. Use them to update guidance. This is how you catch problems before they become incidents.
• Be honest about what you don’t know. The honest answer to most AI questions in schools right now is “we’re still figuring it out.” Say that. Parents and teachers will respect it more than false certainty.

Schools that get AI right aren’t the ones with the most tools or the most ambitious pilots. They’re the ones that move deliberately, protect their students, and build institutional capacity to learn and adjust. That’s not glamorous. But it’s what works.

Frequently Asked Questions

Should schools ban AI tools entirely?

No. Blanket bans are unenforceable and don’t prepare students for a world where AI is embedded in nearly every professional context. The goal isn’t restriction — it’s intentional, informed use. Schools should focus on building the governance structures and staff capacity to adopt AI thoughtfully, not on building walls around it.

What’s the biggest privacy risk with AI tools in schools?

The biggest risk is unauthorized data collection — specifically, student information being used to train AI models without the school’s knowledge or consent. This is especially common with free consumer-grade tools adopted informally by teachers. Any AI tool used with students should have a signed data processing agreement that explicitly prohibits use of student data for model training.

How should schools write an AI policy?

Start with principles, not rules. Define what you’re trying to achieve (student learning outcomes, data protection, academic integrity) and let those principles drive the specific guidelines. Involve teachers, parents, and if age-appropriate, students in the process. Plan to update it annually at minimum — the landscape changes faster than any static document can keep up with. And remember: the policy is only as good as the training and culture that supports it.

What AI tools are actually appropriate for K-8 schools?

Appropriateness depends on your specific use case, student age range, and compliance requirements — there’s no universal answer. That said, purpose-built EdTech tools with clear FERPA/COPPA compliance documentation, signed DPAs, and verifiable age-appropriate content filters are a safer starting point than adapting general-purpose consumer AI tools for classroom use. Always evaluate before you deploy.

How do you help teachers actually use AI well — not just avoid misuse?

The most effective approach I’ve seen is peer learning: identify two or three teachers who are using AI tools thoughtfully and create structured opportunities for them to share their practice with colleagues. This works better than vendor training or top-down PD because it’s grounded in real classroom context. Pair that with clear guidelines on what data can and can’t go into AI tools, and you’ve covered the basics without overwhelming people.

Is AI cheating in schools?

That depends entirely on how the assignment is designed and what the learning goal is. Using AI to generate a first draft you never read or engage with — that undermines the learning. Using AI to check your grammar, brainstorm counterarguments, or understand a concept you’re struggling with — that’s a tool, not a shortcut. Schools need to have explicit, assignment-level conversations about what AI use is expected, permitted, and prohibited. “No AI” and “AI is fine” are both incomplete answers without that context.

—

Rico Tan is the IT Director and AI governance consultant to K-12 schools and nonprofits. He writes about technology, leadership, and institutional decision-making at ricotan.com.