Earthquaker-AI: A RAG Framework with Rubric-Based Evaluation for Elementary Earthquake Education

Background and Context

Traditional earthquake safety education for elementary school students has long struggled to balance the need for interactive engagement with the requirement for deep cognitive processing. Conventional pedagogical methods often rely on passive knowledge transmission, which fails to adequately prepare children for the psychological and logical demands of an actual emergency. Recognizing this gap, researchers have introduced Earthquaker-AI, a hybrid educational framework that integrates educational robotics with Retrieval-Augmented Generation (RAG) technology. This innovation builds upon the award-winning STEM project Earthquaker, which originally utilized Lego WeDo2 mechanical simulations to teach physical responses to seismic events. The new framework extends this foundation from mere mechanical simulation to the levels of cognitive and metacognitive processing, aiming to enhance students' earthquake preparedness and emergency response capabilities through a more sophisticated technological interface.

The core challenge addressed by Earthquaker-AI is the limitation of existing tools in fostering self-regulated learning during crisis scenarios. While physical robots can demonstrate correct actions, they typically lack the capacity to engage students in reflective dialogue or provide nuanced feedback on their decision-making processes. By incorporating a conversational AI assistant, the system shifts the educational paradigm from passive reception to active cognitive construction. This approach not only teaches students what to do but also guides them on how to maintain composure and make correct judgments under pressure. The integration of RAG ensures that the AI's responses are grounded in verified safety guidelines, thereby mitigating the risks associated with unverified generative content in high-stakes educational environments.

Deep Analysis

The technical architecture of Earthquaker-AI employs a synergistic hardware-software design that leverages both physical interaction and digital intelligence. On the hardware side, the system retains the Lego WeDo2 automation platform, utilizing sensors and actuators to simulate earthquake responses. This allows students to engage in embodied cognition, physically interacting with the mechanism to intuitively understand the mechanical principles of protective actions—a dimension that pure software solutions cannot replicate. On the software side, the RAG-driven conversational module serves as the cognitive engine. It semantically matches student queries against official safety guidelines, ensuring that generated answers are both safe and accurate. This dual-layered approach creates a comprehensive learning environment where physical experience is reinforced by intelligent, context-aware feedback.

To accommodate the varying cognitive development levels of elementary students, the system implements a progressive learning trajectory supported by a multi-level rubric evaluation framework. For lower-grade students, the assessment focuses on basic safety action recognition through multiple-choice questions, evaluated using a two-dimensional rubric. Middle-grade students progress to identifying correct action sequences, assessed via a three-dimensional rubric. High-grade students engage in verbal output, writing short answers that are evaluated on a four-dimensional rubric including clarity of expression. This granular evaluation mechanism enables the AI to provide targeted oral feedback, helping students reflect on and optimize their emergency response logic. The rubric-based approach ensures that the feedback is not only corrective but also developmental, aligning with the students' evolving cognitive abilities.

Experimental evaluations of Earthquaker-AI highlight its efficacy in maintaining high standards of content quality and safety. The research team conducted multiple baseline tests to verify the system's performance, focusing on metrics such as grounding, accuracy, and hallucination rates. Grounding refers to the consistency of the AI's responses with official safety guidelines, while accuracy measures the correctness of the information provided. The results demonstrated strong performance in both areas, with the system exhibiting very low hallucination rates. This is particularly critical in educational applications involving life safety, as it ensures that the AI rarely generates misleading or fabricated safety advice. Furthermore, ablation studies, though not detailed in the abstract, confirmed that the hybrid model combining robotics, rubric evaluation, and AI dialogue modules significantly outperforms single-technology approaches in promoting technical literacy and self-regulation.

Industry Impact

The introduction of Earthquaker-AI represents a significant advancement in the intersection of educational technology and public safety. It provides a replicable paradigm for "AI + Public Safety Education," demonstrating how Large Language Models (LLMs) and RAG technology can be effectively deployed to address vertical domain challenges that demand high accuracy. By proving that physical robot interaction can be seamlessly integrated with digital intelligent feedback, the project offers a pathway to bridge the digital divide, allowing young students to acquire complex cognitive skills through embodied interaction. This model challenges the prevailing trend of purely virtual educational tools, suggesting that tangible hardware remains essential for effective early-stage learning in critical safety domains.

Moreover, the rubric-based evaluation framework developed for Earthquaker-AI has broader implications for industrial applications. The methodology can be transferred to other high-risk or high-skill training sectors, such as first aid training or fire safety education, where precise and safe instruction is paramount. The study underscores the potential of technology to foster self-regulated learning and critical thinking, urging developers to look beyond mere answer correctness. Instead, the focus should shift toward guiding the learning process and regulating psychological states. This perspective offers valuable insights for the responsible deployment of AI in education, emphasizing the need for systems that support holistic student development rather than just information delivery.

Outlook

Looking forward, Earthquaker-AI sets a new benchmark for the design of educational AI systems in crisis management contexts. The success of this hybrid framework suggests that future developments in educational technology should prioritize the integration of physical and digital modalities to enhance cognitive engagement. As RAG technology continues to mature, its application in specialized educational domains is likely to expand, offering more robust solutions for ensuring the accuracy and safety of AI-generated content. The emphasis on rubric-based evaluation provides a scalable model for assessing student progress in complex, multi-step tasks, which could be adapted for various STEM subjects beyond earthquake safety.

Additionally, the project highlights the importance of metacognitive training in early education. By encouraging students to reflect on their decision-making processes through AI-mediated dialogue, Earthquaker-AI fosters a deeper understanding of safety protocols that extends beyond rote memorization. This approach aligns with broader educational goals of developing resilient, adaptable learners capable of navigating uncertain environments. As the technology evolves, further research may explore the long-term impact of such hybrid systems on students' emergency response behaviors and psychological resilience. The insights gained from Earthquaker-AI will likely inform the development of next-generation educational tools that are not only technologically advanced but also pedagogically sound and socially responsible.

The implications for the open-source community and educational developers are profound. By providing a transparent framework that combines proven hardware platforms with cutting-edge AI techniques, Earthquaker-AI encourages collaboration and innovation in the field of educational robotics. It serves as a case study for how interdisciplinary research can yield practical solutions to real-world problems. As schools and institutions seek to integrate AI into their curricula, the Earthquaker-AI model offers a viable template for implementing safe, effective, and engaging educational technologies. The continued refinement of this framework could lead to widespread adoption in public safety education, ultimately contributing to a more prepared and resilient society.