What is PRISM and what safety paradox do LLMs face as embodied agent planners?

PRISM is a single-layer L2-regularized logistic probe operating on full hidden states of LLMs to detect physical danger in agent instructions. Research shows content danger and physical danger are separable signals in model representations, invalidating the assumption that textual safety guarantees physical safety.

Why do traditional text moderation and LLM judges fail at physical safety? What are PRISM's advantages?

Text-based methods miss "text-benign but action-dangerous" scenarios. PRISM achieves 86.2%-87.7% accuracy with 11.7%-13.7% false positive rate on SafeAgentBench, significantly outperforming LLM judges with 24.7%-39.0% false positive rates.

What is the significance of PhysicalSafetyBench-1K and the impact of this research?

PSB-1K contains 1000 paired samples without harm keywords to test pure physical risk understanding. PRISM reaches 99.6% accuracy on this benchmark, establishing a new paradigm for representational-level safety detection with low-cost monitoring for high-risk applications like autonomous vehicles.

When Words Are Safe But Actions Are Deadly: Detecting Physical Hazards in Latent State Risk Spaces

Large language models serving as high-level planners for embodied agents face a safety paradox: text-benign instructions can translate into dangerous physical actions. Through hidden-state directional analysis and random-split zero-shot testing, the authors show that content-danger and physical-danger are separable signals in model representations, a phenomenon consistently observed across multiple mainstream models. They propose PRISM, a single-layer L2-regularized logistic probe operating on full hidden states. Experiments demonstrate PRISM achieves 86.2%–87.7% accuracy on SafeAgentBench with extremely low false-positive rates, significantly outperforming同等-sized LLM judges. The study also introduces PhysicalSafetyBench-1K, a benchmark specifically designed to test physical-risk detection without reliance on explicit harm keywords. PRISM nearly perfectly distinguishes safe from dangerous tasks on this benchmark, while traditional LLM judges suffer from severe over-censorship. This work establishes a new paradigm for representational-level physical safety detection that goes beyond text-level content moderation.

Background and Context

The rapid evolution of large language models from passive content generation tools to active high-level planners for embodied agents has fundamentally altered the landscape of AI safety. Traditional safety research has predominantly focused on textual content moderation, targeting hate speech, illegal instructions, or toxic language. However, this text-centric approach fails to account for the physical consequences of agent actions. A critical safety paradox has emerged: instructions that are linguistically benign or even positive can translate into catastrophic physical outcomes when executed by a robot or autonomous system. For instance, a command to "clean the surface" might be interpreted by a planner as using a corrosive chemical if the model lacks an understanding of material compatibility, despite the text itself containing no harmful keywords.

This disconnect highlights a significant gap in current safety paradigms. The assumption that textual safety guarantees physical safety is flawed. The internal representations of large language models do not inherently align linguistic semantics with physical causality. Consequently, models like Qwen2.5, Phi-3.5, and SmolLM2, while proficient in natural language understanding, may generate plans that are textually appropriate but physically hazardous. This phenomenon necessitates a shift from surface-level content filtering to deeper analysis of the model's internal state, where the true intent and potential physical implications of a plan are encoded.

Deep Analysis

To address this challenge, researchers have conducted systematic hidden-state directional analysis and random-split zero-shot testing to investigate the separability of content danger and physical danger within model representations. The core finding is that these two types of risk are distinct signals in the latent space of the model. This separation is consistent across multiple mainstream architectures, indicating that the model's knowledge of linguistic harm is not conflated with its understanding of physical risk. This insight forms the theoretical basis for developing detection mechanisms that operate at the representation level rather than the output level.

Building on this insight, the authors propose PRISM, a novel detection framework designed to identify physical hazards by analyzing the full hidden states of the model. PRISM utilizes a single-layer L2-regularized logistic regression probe that operates directly on the model's internal activations. Unlike traditional methods that rely on the probabilistic outputs of large language models as judges, PRISM employs a discriminative classification strategy. This approach significantly reduces computational overhead and enhances stability. By training on carefully constructed datasets, PRISM learns to map hidden states to physical safety labels, effectively distinguishing between safe and dangerous tasks without relying on generative reasoning.

The design of PRISM emphasizes generality and efficiency. It does not require fine-tuning for specific model architectures, making it a versatile tool for various large language models of different scales. This lightweight probe can be deployed to monitor embodied agents in real-time, providing a cost-effective solution for safety assurance. The method's ability to capture subtle differences in internal representations allows it to detect risks that are invisible to text-based filters, particularly in scenarios where the danger lies in the physical execution rather than the linguistic content.

Industry Impact

Experimental evaluations demonstrate the superior performance of PRISM across several benchmarks. On SafeAgentBench, PRISM achieved an accuracy range of 86.2% to 87.7%, while maintaining a low false-positive rate between 11.7% and 13.7%. In contrast, equivalent-sized LLM judges, such as Qwen2.5-3B, exhibited significantly higher false-positive rates, ranging from 24.7% to 39.0%. This discrepancy indicates that traditional LLM-based judges are prone to over-censorship, frequently flagging safe instructions as dangerous due to a lack of nuanced understanding. The high false-positive rate in traditional methods can lead to inefficient agent operations and reduced trust in automated systems.

To further test the sensitivity of PRISM to physical risk rather than keyword dependence, the study introduced PhysicalSafetyBench-1K (PSB-1K). This benchmark consists of 1,000 pairs of physical risk comparison samples that do not contain explicit harm keywords. PSB-1K is designed to evaluate whether a model truly understands the causal relationships in the physical world. On this benchmark, PRISM achieved an impressive accuracy of 99.6% with a false-positive rate of only 0.7%. Conversely, the Qwen2.5-3B judge incorrectly rejected 67.8% of safe tasks, highlighting the severe limitations of text-based moderation in physical safety contexts.

The introduction of PSB-1K fills a critical void in the field of physical safety evaluation, providing a standardized platform for future research. The robust performance of PRISM on SafeText and EAR-Bench further confirms the broad applicability of hidden-state probing as a representation-level method. These results underscore the importance of moving beyond text-level content moderation to establish a dedicated physical safety assessment framework, which is essential for the safe deployment of embodied AI in real-world environments.

Outlook

This research has profound implications for both the open-source community and industrial applications. It challenges the prevailing text-centric safety paradigm, advocating for the development of independent physical safety evaluation systems tailored to the era of embodied intelligence. For industry, PRISM offers a low-cost deployment strategy that enables real-time monitoring of potential risks without the need to retrain large foundation models. This capability is crucial for high-stakes applications such as autonomous driving and home service robots, where safety failures can have severe consequences.

Furthermore, the discovery of the separability between content danger and physical danger provides new perspectives on understanding the internal knowledge representations of large language models. This finding may inspire future research into model interpretability and alignment techniques, potentially leading to more robust and trustworthy AI systems. By establishing a new paradigm for representational-level physical safety detection, this work paves the way for more reliable human-machine interaction. As embodied agents become increasingly integrated into daily life, the ability to detect and mitigate physical hazards at the representation level will be a cornerstone of safe and effective AI deployment.

The transition from textual safety to physical safety represents a fundamental shift in AI governance. As models grow more capable of planning and executing complex tasks, the reliance on simple keyword filtering will become increasingly inadequate. The PRISM framework and the PhysicalSafetyBench-1K benchmark provide a scalable and effective solution to this growing challenge. By focusing on the internal states of the model, researchers and developers can gain deeper insights into the decision-making processes of AI agents, ensuring that their actions align with human values and physical realities. This approach not only enhances safety but also fosters the development of more intelligent and responsive embodied systems.

Sources

arXiv