What is EAGLE-360 and what problem does it solve?

EAGLE-360 is a new framework enabling multimodal large language models to perform active visual search in 360° panoramic environments. It tackles the challenges of polar distortion and inefficient local search by abandoning fragmented local approaches in favor of a global prior-driven iterative reasoning strategy that progressively narrows the search space.

Why does EAGLE-360 matter? What are its practical implications?

EAGLE-360 achieves state-of-the-art performance on 360° visual search, improving target detection accuracy by nearly 8× over baselines. Its approach offers viable technical paths for autonomous driving perception, VR/AR navigation, and robot environmental understanding, reducing computational load while improving response speed.

What should we watch for next regarding EAGLE-360?

The team has released a large-scale dataset of 14,000 4K panoramic images and 70,000 high-quality VQA dialogues, filling a critical data gap. Future directions include extending RoPE Rolling to other complex-topology vision tasks and scaling the SFT+GRPO training paradigm for embodied intelligence in broader 3D environments.

EAGLE-360：基於全局先驗的360°全景主動探索與視覺搜尋框架

針對多模態大語言模型在360°全景環境中主動視覺搜尋時面臨的極區畸變建模困難及局部搜尋效率低下的問題，本文提出EAGLE-360框架。該方法摒棄傳統碎片化局部搜尋，利用全局先驗建立整體視角，通過迭代推理逐步縮小搜尋空間。技術上創新適配RoPE Rolling機制以處理全景圖的連續圓柱拓撲結構，並結合SFT與GRPO訓練策略強化空間推理與工具呼叫能力。同時構建包含1.4萬張4K全景圖及7萬輪高品質VQA對話的大規模數據集。實驗表明，EAGLE-360在360°視覺搜尋任務上取得最新最優性能，目標檢測準確率較基線模型提升近8倍，顯著改善探索效率與錯誤恢復能力。

Sources

arXiv