How does SAM 3 perform on remote sensing imagery?

SAM 3 achieves high F1-scores in segmentation tasks while avoiding overfitting problems of traditional domain adaptation models. However, visual prompts align well with overhead geometries while text prompts introduce ground-level semantic biases, causing severe cross-modal interference.

Why does SAM 3 matter for Earth observation?

The study repurposes SAM 3's binary existence head as an independent zero-shot classifier, enabling training-free scene classification. Yet it remains limited by sub-pixel resolution and semantic blind spots, struggling with small or dense objects—direct deployment alone is insufficient for high-precision remote sensing.

What should be the next step?

Parameter-efficient fine-tuning (PEFT) of the multi-modal decoder is urgently needed to resolve geometric-semantic misalignment. The proposed training-free proxy evaluation protocol also offers a standardized assessment method for other foundation models in vertical domains.

SAM 3遙感零樣本與少樣本能力評估：提示機制與跨模態干擾深度解析

本文針對 Segment Anything Model 3（SAM 3）在地球觀測遙感影像中的泛化能力進行了全面評估。研究指出，儘管 SAM 3 旨在實現開放詞彙的免訓練計算機視覺，但其在處理遙感圖像複雜俯視幾何結構時的表現尚不明確。作者透過嚴格的零樣本和單樣本約束，在場景分類、目標檢測和實例分割等多任務中評估了 SAM 3 的效能。研究提出了一種結構適配方法，將 SAM 3 解耦的二進制存在頭重新用作獨立的零樣本分類器，並透過隔離文本與視覺提示模態，診斷了多模態解碼器中的對齊機制。實驗發現存在嚴重的跨模態干擾：視覺提示能有效對齊複雜幾何結構，而文本提示則引入地面級語義偏差，降低座標回歸精度。此外，研究制定了免訓練的代理評估協議，結果顯示 SAM 3 避免了傳統領域適配模型的過擬合問題，在分割任務中取得高調和均值，但仍受限於亞像素解析度與語義盲點，亟需參數高效的微調方案。

Sources

arXiv