Construction of AI Learning Support Model in Response to Conversation Content and Emotional Changes by EQ

Abstract

Instructors generally find it difficult to intuitively grasp learners' emotions and level of understanding. Therefore, there is an increasing need for individualized instruction that provides support tailored to the student's emotional state and level of understanding. To objectively assess learner states, AI-based learning support grounded in physiological indicators that interpret emotional changes is indispensable. This study estimates learner states in data science education based on emotional changes detected through conversation content and EQ. After state estimation, the study proposes a method that constructs an AI-based learning support model according to learner states. It identifies appropriate instructional strategies for instructors. Instructor shortages and the difficulty of providing individually optimized support have become apparent. This study collects conversational data and electrodermal activity to estimate learner states. A hidden Markov model (HMM) estimates learners’ internal states from the conversational behaviors of instructors and learners. This study conducts one-on-one tutoring sessions between an instructor and a learner to evaluate the effectiveness of the proposed method. The experimental results reveal three states: a trial and error state, a state of searching for understanding, and an initial state of reaching understanding. The above results indicate that learner states can generally be estimated from conversation content. The study also constructs a Random Forest model based on the estimated learner state and conversation content. The F1 score is 0.824, enabling the identification of key features strongly associated with each learning state. Furthermore, the study fine-tunes a BERT model using utterance-level dialogue data annotated with state labels to classify learner states. The F1 score of 0.9015 indicates high accuracy, demonstrating the model's ability to accurately estimate the learner's state based on conversation content. Analyzing the state transitions, the study can help instructors decide on effective teaching methods. The findings obtained in the study hold promise for enhancing the quality of individualized instruction and as a model for autonomous learning support through AI agents.