AI Topic Models
Client: Conceptual Academy
Conceptual Academy's AI tutoring system, Alia, successfully engaged students in physics and chemistry education but revealed a new challenge: How could teachers meaningfully interpret the wealth of conversation data generated by these interactions? Traditionally, understanding what topics students discussed with an AI tutor would require an extensive manual review of chat logs or complex topic modeling that needed data science expertise to interpret. Teachers lacked a way to quickly identify what concepts their students were struggling with without sifting through individual conversations or requiring advanced analysis skills. This insight gap limited the pedagogical value of the conversation data and prevented teachers from effectively targeting their instruction to address student knowledge gaps
Delphi Intelligence developed an innovative topic modeling dashboard that transforms complex student-AI interactions into clear, actionable insights. The solution addresses the interpretability challenges of traditional approaches. Rather than presenting difficult-to-interpret PCA plots and raw word weights, the system feeds this metadata, along with representative sample documents, to a Large Language Model. This LLM generates human-readable topic names, key bullet points, and summary paragraphs that contextualize conversation themes. The dashboard presents this information alongside interactive word clouds and topic distribution visualizations, all segmented by classroom to ensure relevance.
The implementation of the Alia Topic Modeling Dashboard has fundamentally transformed how teachers interface with AI-assisted learning data. Educators can now immediately identify which concepts their students are struggling with without needing data science expertise. This capability enables teachers to proactively adjust their instruction to address specific knowledge gaps, creating a feedback loop between AI tutoring and classroom teaching. The classroom-specific segmentation ensures privacy while providing targeted insights to individual teachers. By solving the critical interpretation gap that has historically limited the usefulness of topic modeling in educational settings, Delphi Intelligence has made advanced NLP techniques accessible and actionable for non-technical stakeholders, significantly enhancing Conceptual Academy's AI tutoring platform.
