What is PRISM and how does it detect physical safety risks in LLMs?

PRISM is a lightweight single-layer L2-regularized logistic probe that operates on an LLM's full hidden states. Unlike traditional text-based safety evaluators, it predicts physical danger from internal model representations before action instructions are generated, without needing to retrain the base model.

Why does LLM safety alignment need to go beyond text content risks?

As LLMs serve as high-level planners for embodied agents, textually benign commands can become dangerous when executed physically — a robot clearing a desk might knock over fragile or hazardous items. Research shows physical and content risks form separable signals in model representations.

What impact could PRISM and PhysicalSafetyBench-1K have on embodied AI safety?

PRISM achieves 86.2%-87.7% accuracy with only 11.7%-13.7% false positives on SafeAgentBench. The new PhysicalSafetyBench-1K benchmark of 1,000 physical risk pairs provides a standardized testbed and could extend to multimodal safety for vision and touch.

當文字安全但行動致命：探索隱藏狀態風險空間中的物理危險

大型語言模型正逐漸作為具身智能體的高階規劃器，但語言上無害的指令在物理世界落地時可能產生危險。本文深入探討了這種基於物理環境的危險是否等同於普通的文本內容危險。透過對 Qwen2.5、Phi-3.5 和 SmolLM2 等多個模型的隱藏狀態方向分析和隨機分割零測試，研究發現內容危險與物理危險在模型表示中形成了可分離的訊號。基於這一發現，作者提出了 PRISM，一種基於全隱藏狀態的單層 L2 正規化邏輯探針。實驗顯示，PRISM 在 SafeAgentBench 上達到了 86.2% 至 87.7% 的準確率，且誤報率僅為 11.7% 至 13.7%，顯著優於同等規模的 LLM 評判器。此外，本文引入了包含 1,000 個物理風險對的 PhysicalSafetyBench-1K 基準測試，進一步驗證了該方法能有效識別基於物理環境的危險而非僅依賴不安全詞彙。

Sources

arXiv