Fig. 1

Overview of the Proposed AmmH Model. AmmH is composed of two hybrid single-modal encoders, an adaptive cross-modal encoder, and a MLP Head. Two single-modal encoders are used to extract high-level features from US images and IRT images respectively, an adaptive cross-modal encoder is used for feature fusion and the MLP Head uses the resulting features for the final thyroid nodules classification. Besides, a modality-weight generation network generates different modal weights \((\omega _{US},\omega _{IRT})\) for each case based on the features from the single-modal encoders