Towards second opinion: anatomy-aware reasoning from explicit priors to adaptive gating for efficient medical image analysis

Abstract

This study presents SecondOpinion, a dual-stream deep learning framework for medical image analysis that dynamically adjusts computational effort according to case difficulty. A lightweight GateKeeper module, trained as a binary correctness classifier, determines whether a primary prediction is reliable or whether an additional anatomy-guided stream with cross-attention should be activated. Evaluated on chest X-ray disease classification and pelvic fracture detection, the framework achieves performance close to an always-on model while substantially reducing computational costs. The results show that conditional gating can preserve diagnostic performance, reduce FLOPs, and activate additional reasoning more frequently for diagnostically difficult cases, including invisible fractures. The study demonstrates the potential of adaptive computation as a decision-support approach for efficient medical image analysis.