Salman Sayeed

The development of Pedagogical Conversational Agents (PCAs) made significant progress due to the rise of Large Language Models (LLMs). Yet, creating and evaluating PCAs remains a technically challenging task for educators. Our work presents a semi-autonomous system, NuevAI; designed to democratize the access to AI-enhanced educational tools by simplifying the development of PCAs. NuevAI consists of two complementary platforms that help educators create structured, multi-turn conversational datasets from various data sources for fine-tuning LLMs and also facilitates systematic assessment of PCA conversations by learners. From the evaluation of 16 educators from STEM subjects, our system shows notable improvements compared to traditional methods: 74.5% usability score (vs. 50.9% baseline), 81.2% task efficiency (vs. 49.1%), and 80.8% time efficiency (vs. 32.5% baseline). The system achieved a Net Promoter Score of 68.8% (vs. -62.5% baseline) and reduced cognitive workload by 40% as measured by NASA-TLX assessment. By providing user-friendly and intuitive interfaces and automated workflows, NuevAI empowers educators to create adaptive learning tools through PCAs, aligned with specific pedagogical objectives without requiring technical expertise in the field of AI.

The rapid advancement of Large Language Models (LLMs) has opened new opportunities to transform various aspects of human life, including education. This thesis explores the integration of LLMs into teaching through the development of Pedagogical Conversational Agents (PCAs). We incorporate key capabilities such as vector search, image retrieval, code-generated animations, and supervised fine-tuning to create effective PCAs tailored to specific pedagogical objectives. The system that we propose, helps learners acquire core knowledge and deeper conceptual understanding with hint-based mathematical and analytical problem solving. Our approach is applied to K-12 physics, and we evaluate its effectiveness using multiple assessment frameworks. To ensure accessibility for educators with minimal technical expertise, we also implement a dual-platform system that enables non-technical K-12 teachers to create high-quality datasets, fine-tune models, and conduct human evaluations of the generated PCAs with students.