Research update: Inside the AI for Advancing Instruction at Scale project
The pilot phase is officially underway for the AI for Advancing Instruction at Scale (AI2S) project.
This research-driven project represents a major leap in educational technology: a system that watches and listens to classroom videos to provide intelligent, actionable feedback and coaching, specifically tailored for mathematics instruction.
Teachers in districts from Texas, New York, and Virginia will use AI2S this year, which combines cutting-edge computer vision and audio analysis to give teachers the most comprehensive picture of their instruction to date.
Designed to support educators in delivering rigorous math instruction while lowering costs for schools and districts, the project is being developed in collaboration with researchers from the University of Virginia and the University at Albany, with funding from the Gates Foundation.
We’re excited to share what we’ve accomplished so far and give you a closer look at how this AI-powered assistant was designed with teachers’ real needs in mind.
How it works: A virtual coach that sees and hears teaching
AI2S is designed to make the complicated work of analyzing and improving instruction simple, fast, and effective.
The system analyzes classroom dynamics and student engagement patterns while listening to teacher-student dialogue, questioning patterns, and instructional language.
For example, teachers receive data about:
- Proximity to students
- Student participation (e.g., peer-to-peer, hand raising)
- Question types (Open-ended, closed-ended, or task-related)
- Academic language, and more!
Here’s how the coaching experience works for teachers:
- Upload video for analysis. Teachers upload their classroom video, and AI2S analyzes it.
- Set the focus. Guided by the virtual coach, teachers identify an aspect of math instruction they want to improve.
- See the big picture. Before diving directly into specific moments, teachers review a high-level overview of classroom dynamics and instructional patterns related to their focus area.
- Dive into details. Based on responses to personalized questions, the system selectively reveals data to help teachers explore their instruction more deeply and meaningfully.
- Reflect and plan. After analyzing the data, teachers step back from the metrics to reflect on what they’ve learned. With support from the virtual coach, they develop a concrete action plan to enhance instruction.
With reflection and planning complete, teachers return to the classroom equipped with data-informed insights, specific strategies, and a clear roadmap for improvement.
What makes this AI different for improving teaching? Audio AND video analysis
“There are limitations with current AI models in the classroom,” explains Jonathan Foster, assistant professor of mathematics education at the University of Albany. “A lot of [existing research projects] are just audio-based, so it will do some speech-to-text recognition and analyze the resulting classroom transcript. But it’s often missing other modalities, like the video itself.”
AI2S changes this. The system is trained on classroom videos that have been hand-coded by expert human observers, creating a unique dataset that enables reliable, credible analysis of uploaded classroom videos. The neural network is already research-validated and proven to be as reliable as a human observer.
This training allows the AI ssystem to:
- Detect the quality of teacher-student mathematical dialogue.
- Assess the cognitive rigor of lessons through both verbal and visual cues.
- Recognize instructional patterns that emerge from the combination of what teachers say and do.
- Identify student engagement through verbal and non-verbal signals.
By integrating this neural network into the AI Coach experience, teachers don’t need to be data scientists to understand their classroom data. Instead, with the virtual coach’s guidance, they can analyze their practice and make improvements with confidence.
Think of this tool as having an expert observer who never gets tired, never misses details, and can identify patterns that human observers might overlook, but who still supports professional judgment rather than replacing it.
How the research is being conducted
This isn’t just a pilot program – it’s a comprehensive research study. Teachers from districts across three states are participating, contributing to our understanding of how AI can best support mathematics instruction when it has access to the full complexity of classroom interactions.
Key components of the study:
- Teachers complete four coaching cycles throughout the year
- Researchers gather quantitative and qualitative insights from each cycle
- Analysis includes:
- Feedback quality
- Changes in teaching practices
- Teacher noticing patterns
- Scoring based on the MQI framework
In addition, researchers will explore how teachers experience the tool and will work with them to understand how to make the experience even more effective.
A core objective of this independent research project is to determine how video-based AI coaching improves teaching and learning. This goes well beyond simply integrating AI into classrooms with the assumption of positive impact.
In the end, schools will have a trusted, research-validated tool that leverages the full richness of classroom video to help teachers improve math instruction, not just more AI hype.
If your school or district is interested in participating in this research project, we’d love to hear from you.
Built with teacher input: The co-design process
Before writing a single line of code, the research team conducted extensive co-design sessions with educators and instructional coaches. These sessions surfaced pain points and feature requests that shaped the system’s design.
What we learned:
- AI-based feedback can be appealing in theory, but it often lacks depth when based only on transcripts.
- Teachers find many dashboards overwhelming and hard to act on.
- Useful data should be delivered step-by-step and in context, not all at once.
We asked educators to participate in card sort exercises like the one above to help us understand how educators aligned preferences for the available metrics.
In response to this real teacher and coach feedback, we are able to prioritize which metrics show up and when. In addition, rather than showing all of the data all at once, the tool essentially offers teachers a guided tour of their data, taking them through each focus area step-by-step.
This approach ensures the feedback is both digestible and actionable, helping educators focus on meaningful growth rather than feeling buried in data.
Teachers are still in the driver’s seat
Every part of this platform reflects our belief that teachers should be in charge of their own professional development.
The tool doesn’t dictate what teachers should do. Instead, it provides detailed, targeted data that supports reflection and improvement. Teachers decide what to change and how to apply insights in their classroom.
This approach recognizes that effective teaching requires human judgment, creativity, and care, qualities that no algorithm can replace.
Instead, AI2S acts as a thinking partner, offering expert-level analysis of both verbal and visual classroom dynamics, so educators can make informed, empowered decisions.
As UVA’s Scott Acton notes, “This is about helping teachers do what they do best, teach, while making professional development more accessible and equitable.”
Join the excitement
You don’t need to wait for the results of the study to leverage the benefits of AI-enabled coaching today. The AI2S tools are built on top of our existing AI Coach platform experience, which provides teachers with support for core teaching practices and structured literacy. Teachers can work alongside the virtual coach to analyze their teaching, develop an action plan, and measure impact.
Ready to learn more about AI Coach and bring it to your teachers? Contact our team to explore how to get started.
