MEOX: A Compact Multimodal Mixture-of-Experts Model for Earth Observation

Background and Context

The field of Earth observation has long been constrained by the inherent heterogeneity of sensor data and the frequent absence of observations across different spectral bands. Traditional deep learning solutions have attempted to mitigate these challenges by constructing massive architectures capable of accommodating complex multimodal inputs. However, this reliance on large-scale models introduces significant computational burdens, severely limiting their deployment in resource-constrained environments such as edge devices or satellite terminals. To address this critical bottleneck, researchers have introduced MEOX, a compact multimodal masked autoencoder specifically engineered for Earth observation tasks. This model represents a paradigm shift by demonstrating that high-performance representation learning does not necessitate vast parameter counts.

MEOX achieves efficient cross-modal representation learning with an encoder comprising only 2.939 million parameters, bringing the total model size to 3.115 million. This extreme compactness is made possible through a novel architectural design that integrates sensor-specific adapters, explicit validity signals, and shared sparse expert blocks. By preserving modality dependencies before performing fusion at the patch level, the model effectively handles the disparities introduced by heterogeneous sensors. Furthermore, it employs structured strategies to manage missing data, offering a lightweight solution that reduces the trade-off between model complexity and performance. This approach allows for robust feature extraction without the overhead associated with conventional large-scale multimodal systems.

Deep Analysis

The technical architecture of MEOX is built upon a sophisticated network structure designed to ensure efficient information fusion and feature extraction. The process begins with sensor-specific adapters that process input data, augmented by explicit validity signals to identify data reliability. These inputs are then processed through shared sparse expert blocks, which maintain modality independence prior to patch-level fusion. The fused sequence is accompanied by four metadata tokens and undergoes deep processing via fourteen additional encoder blocks. To constrain parameter growth while enhancing generalization, the model utilizes a strategy combining shared expert projections with private low-rank residuals. This design ensures that general knowledge is shared across modalities while preserving specific nuances unique to individual sensors.

A key innovation within MEOX is the integration of Rotary Attention, which enables the model to support spatial layouts in downstream tasks that differ from those encountered during pre-training. This flexibility significantly enhances the model's adaptability to varying spatial grids. The model was pre-trained on a large-scale dataset of 1.228 million MMEarth64 samples, employing modality-balanced masked reconstruction and structured sensor dropout strategies. These training methods were selected to improve robustness against missing data and sensor variability. The routing diagnostics further reveal critical internal mechanisms, including expert participation patterns, spatial dependencies, and modality correlations, providing deep insights into how the model allocates computational resources to different types of Earth observation data.

Industry Impact

Experimental results validate the superior performance of MEOX across multiple benchmarks, demonstrating its capability to outperform existing models despite its minimal size. In frozen transfer evaluations on the GEO-Bench benchmark, MEOX achieved a mean Intersection over Union (IoU) of 64.42% at 64-pixel resolution for segmentation tasks. At a higher resolution of 224 pixels, the model attained an average accuracy of 90.56% on the EuroSAT classification task. Both metrics surpass the results reported by previous CSMoE models, highlighting the efficiency of the proposed architecture. Additionally, fine-tuning experiments on the BigEarthNet dataset yielded a micro-average precision of 72.95%, further confirming the model's strong generalization capabilities and effectiveness in complex classification scenarios.

The implications of MEOX extend beyond academic benchmarks to practical industrial applications. Its compact parameter footprint enables deployment on edge devices and resource-limited environments, facilitating real-time monitoring and rapid response capabilities in Earth observation systems. By proving that sophisticated multimodal fusion can be achieved without massive parameter budgets, MEOX sets a new standard for efficient AI in remote sensing. The model's emphasis on metadata utilization and cross-sensor alignment underscores the importance of integrating diverse data sources to enhance task performance. This approach encourages the broader community to explore deeper integration mechanisms between metadata and visual features, potentially leading to more robust and versatile Earth observation tools.

Outlook

MEOX represents a significant advancement in the development of lightweight, intelligent tools for Earth observation. Its success in achieving high performance with minimal parameters challenges the prevailing trend of scaling up model sizes and suggests a more sustainable path for future research. The open-source nature of the model is expected to accelerate innovation within the community, allowing researchers and developers to build upon its efficient architecture. As the demand for real-time, on-device processing in remote sensing grows, models like MEOX will likely become foundational components in next-generation Earth observation systems.

Looking forward, the mechanisms identified through routing diagnostics, such as expert participation and modality correlations, offer valuable directions for further optimization. Future work may focus on expanding the model's capability to handle even more diverse sensor types or integrating additional metadata sources to refine spatial and temporal predictions. The structured sensor dropout strategy employed during pre-training also provides a template for handling incomplete data in other domains. Ultimately, MEOX demonstrates that compact, multimodal architectures can deliver state-of-the-art results, paving the way for more accessible and efficient AI solutions in critical environmental monitoring and analysis tasks.