ARC-CT: Contrastive Vision-Language Learning with Anatomical Routing for 3D Chest CT Analysis

Background and Context

The integration of chest computed tomography (CT) volumes with radiological reports has emerged as a pivotal pathway for training abnormality classifiers without the need for manual annotation. This approach leverages contrastive vision-language learning to align visual data with textual descriptions, offering a scalable solution for medical image analysis. However, traditional global contrastive learning methods encounter significant hurdles when applied to 3D chest CT data. The primary challenge lies in the dilution of local lesion features. Many critical pathological abnormalities manifest as subtle or anatomically localized characteristics. When a complete 3D volume is pooled and compressed into a single embedding vector, these fine-grained visual evidences are often lost, preventing the model from capturing key病灶 details effectively.

A second major limitation involves the handling of negative samples in standard contrastive objectives. Typically, all other scans in a batch are treated as negative examples. In clinical practice, however, many chest CT scans share identical abnormal features. Treating these similar positive samples as negatives imposes a false-negative penalty, pushing the model to separate samples that should be clustered together. This misalignment hinders the model's ability to learn robust representations of shared pathologies. To address these specific limitations, the research introduces ARC-CT, a framework designed to operate without human annotations or bounding boxes, relying solely on labels extracted from reports.

Deep Analysis

ARC-CT is built upon a sophisticated architecture comprising three core components that enable fine-grained visual-language alignment. The first is AnatomyQFormer, which employs a query mechanism constrained by automatically generated organ masks. This allows the model to focus on specific anatomical regions, locating key evidence within the 3D volume rather than relying on global pooling. By restricting queries to organ-level masks, the system ensures that visual features are grounded in relevant anatomical structures, enhancing the precision of evidence localization.

The second innovation is the label-Jaccard soft InfoNCE objective function. This design merges the standard one-to-one contrastive target with the overlap degree of label sets between sample pairs. By calculating the Jaccard similarity of labels, the model can identify scan pairs that share clinical findings. This mechanism significantly reduces the false-negative penalty for samples with common positive features, preventing the erroneous separation of similar lesions. It effectively mitigates the negative interference caused by shared lesion samples, a common issue in traditional contrastive learning frameworks.

The third component is the organ-level alignment loss, which connects mask-pooled visual features with specific organ report texts extracted offline using large language models. This multi-level alignment strategy ensures that the model matches not only in overall semantics but also in precise correspondence between anatomical structures and pathological descriptions. This approach improves the robustness and interpretability of the feature representations, allowing the system to provide more reliable diagnostic insights without requiring explicit pixel-level annotations.

Industry Impact

The experimental evaluation of ARC-CT demonstrates its superior performance across a dataset containing 18 common abnormalities. Using an unmasked macro AUC as the key metric, the framework achieved a score of 0.86. Notably, this result was obtained using a lightweight 3D ResNet-18 backbone, which requires significantly fewer parameters and computational resources compared to many existing large-scale models. Despite its compact size, ARC-CT outperformed various efficient baselines and surpassed several larger Transformer models in diagnostic accuracy. This efficiency makes it highly suitable for deployment in resource-constrained environments, such as hospitals with limited computing infrastructure.

Ablation studies confirmed the effectiveness of each component, particularly the role of the label-Jaccard objective in reducing false-negative penalties and the advantage of AnatomyQFormer in extracting local features. The open-source release of ARC-CT’s code and weights provides a valuable resource for the research community, facilitating further exploration in contrastive learning and medical AI. By lowering the barrier to data preparation through automated label extraction, the framework enables the utilization of large-scale medical data without the prohibitive costs of manual annotation.

Outlook

ARC-CT offers a practical and scalable solution for medical image analysis, bridging the gap between high performance and computational efficiency. Its region-aware characteristics provide a new direction for improving model interpretability and diagnostic accuracy through anatomical routing mechanisms. As healthcare systems increasingly seek to integrate AI into clinical workflows, the ability to provide transparent and trustworthy diagnostic support is crucial. ARC-CT’s design fosters trust by aligning model decisions with specific anatomical evidence, making it a promising tool for assisting radiologists in rapid screening and diagnosis.

Looking forward, the framework’s reliance on LLM-extracted labels suggests potential for further automation in data preprocessing. Future research may explore extending this approach to other modalities or more complex multi-organ analyses. The success of ARC-CT in achieving high accuracy with lightweight architectures underscores the importance of efficient model design in medical AI. As the field moves towards more integrated and accessible diagnostic tools, frameworks like ARC-CT will play a vital role in democratizing advanced image analysis capabilities, ensuring that high-quality diagnostic support is available across diverse healthcare settings.