MetaBreastAI
An Explainable Dual-Stream Convolutional Neural Network-Transformer Framework with Multi-Instance Learning for Breast Cancer Metastasis Detection
DOI:
https://doi.org/10.14500/aro.12768Keywords:
Breast Cancer Metastasis, Convolutional neural network-transformer hybrid, Dual-stream deep learning, Medical image classification, Multi-instance learningAbstract
Breast cancer metastasis is one of the most consequential clinical problems in breast cancer, as it is responsible for most of the breast cancer attributable deaths in women globally. Timely detection of metastasis progression is crucial for improving therapies and enhancing patient survival. Although deep learning approaches, especially convolutional neural networks (CNNs), still fail to model global dependencies, they cannot work under weak supervision, and unable to generate interpretable predictions. Transformer-based models, on the other hand, provide a deeper contextual knowledge; but they are insensitive to local patterns and require large datasets. To overcome these issues, we present MetaBreastAI, an explainable dual-stream deep learning framework comprising a CNN branch fused with the convolutional block attention module and a Transformer stream. Each of these parallel branches captures spatial and contextual features, respectively, which are fused by a multi-instance learning approach for partially supervised classification. The proposed model has been tested on benchmark computed tomography scan dataset, it enables identifying one of the distinct features of MetaBreastAI and apply feature attribution methods to both visual and quantitative experiments, demonstrating that MetaBreastAI achieves better performance (macro-averaged F1-score: 91.7%; area under the curve: 96.1%) compared to each branch independently, and hybrid baseline models. By highlighting the lesion’s location, the heatmap supports interpretability. Due to the explainable detection of lesions in metastatic cases, our hybrid model provides a scalable and clinically feasible strategy through overcoming the downsides of earlier research, then it improves the reliability of AI-assisted techniques in medical decision-making.
Downloads
References
Abdollahi, J., Davari, N., Panahi, Y., and Gardaneh, M., 2022. Detection of metastatic breast cancer from whole-slide pathology images using an ensemble deep-learning method. Archives of Breast Cancer, 9, pp.364-376. DOI: https://doi.org/10.32768/abc.202293364-376
Abhisheka, B., Biswas, S.K., and Purkayastha, B., 2023. A comprehensive review on breast cancer detection, classification and segmentation using deep learning. Archives of Computational Methods in Engineering, 30, pp.5023-5052. DOI: https://doi.org/10.1007/s11831-023-09968-z
Afonso, M., Bhawsar, P.M.S., Saha, M., Almeida, J.S., and Oliveira, A.L., 2024. Multiple instance learning for WSI: A comparative analysis of attention-based approaches. Journal Pathology Informatics, 15, p.100403. DOI: https://doi.org/10.1016/j.jpi.2024.100403
Ahmad, S., Ullah, T., Ahmad, I., Al-Sharabi, A., Ullah, K., Khan, R.A., Rasheed, S., Ullah, I., Uddin, M.N., and Ali, M.S., 2022. A novel hybrid deep learning model for metastatic cancer detection. Computational Intelligence and Neuroscience, 2022, p.8141530. DOI: https://doi.org/10.1155/2022/8141530
Allugunti, V.R., 2021. Breast cancer detection based on thermographic images using machine learning and deep learning algorithms. International Journal of Engineering in Computer Science, 41, pp.49-56. DOI: https://doi.org/10.33545/26633582.2022.v4.i1a.68
Bandi, P., Geessink, O., Manson, Q., Van Dijk, M., Balkenhol, M., Hermsen, M., Bejnordi, B.E., Lee, B., Paeng, K., Zhong, A., Li, Q., Zanjani, F.G., Zinger, S., Fukuta, K.,… & Litjens, G., 2018. From detection of individual metastases to classification of lymph node status at the patient level: The CAMELYON17 challenge. IEEE Transactions on Medical Imaging, 38, pp.550-560. DOI: https://doi.org/10.1109/TMI.2018.2867350
Barbosa, D., Ferreira, M., Junior, G.B., Salgado, M., and Cunha, A., 2024. Multiple instance learning in medical images: A systematic review. IEEE Access, 12, pp.78409-78422. DOI: https://doi.org/10.1109/ACCESS.2024.3403538
Botlagunta, M., Botlagunta, M.D., Myneni, M.B., Lakshmi, D., Nayyar, A., Gullapalli, J.S., and Shah, M.A., 2023. Classification and diagnostic prediction of breast cancer metastasis on clinical data using machine learning algorithms. Scientific Reports, 13, p.485. DOI: https://doi.org/10.1038/s41598-023-27548-w
Chakraborty, D., Ivan, C., Amero, P., Khan, M., Rodriguez-Aguayo, C., Basagaoglu, H., and Lopez-Berestein, G., 2021. Explainable artificial intelligence reveals novel insight into tumor microenvironment conditions linked with better prognosis in patients with breast cancer. Cancers (Basel), 13, p.3450. DOI: https://doi.org/10.3390/cancers13143450
Das, A., Narayan Mohanty, M., Kumar Mallick, P., Tiwari, P., Muhammad, K., and Zhu, H., 2021. Breast cancer detection using an ensemble deep learning method. Biomedical Signal Processing and Control, 70, p.103009. DOI: https://doi.org/10.1016/j.bspc.2021.103009
Fu, B., Zhang, M., He, J., Cao, Y., Guo, Y., and Wang, R., 2022. StoHisNet: A hybrid multi-classification model with CNN and Transformer for gastric pathology images. Computer Methods and Programs in Biomedicine, 221, p.106924. DOI: https://doi.org/10.1016/j.cmpb.2022.106924
Fu, F., Zhang, X., Wang, Z., Xie, L., Fu, M., Peng, J., Wu, J., Wang, Z., Guan, T., He, Y., Lin, J.S., Zhu, L., and Dai, W., 2025. A pathology-attention multi-instance learning framework for multimodal classification of colorectal lesions. Frontiers Pharmacology, 16, p.1592950. DOI: https://doi.org/10.3389/fphar.2025.1592950
Hany, M., 2021. Chest CT-Scan Images Dataset. Kaggle, United States.
Hossain, M.S., Shahriar, G.M., Syeed, M.M.M., Uddin, M.F., Hasan, M., Shivam, S., and Advani, S., 2023. Region of interest (ROI) selection using vision transformer for automatic analysis using whole slide images. Scientific Reports, 13, p.11314. DOI: https://doi.org/10.1038/s41598-023-38109-6
Hu, W., Li, X., Li, C., Li, R., Jiang, T., Sun, H., Huang, X., Grzegorzek, M., and Li, X., 2023. A state-of-the-art survey of artificial neural networks for whole-slide image analysis: From popular convolutional neural networks to potential visual transformers. Computers in Biology and Medicine, 161, p.107034. DOI: https://doi.org/10.1016/j.compbiomed.2023.107034
Ikromjanov, K., Bhattacharjee, S., Hwang, Y.B., Sumon, R.I., Kim, H.C., and Choi, H.K., 2022. Whole Slide Image Analysis and Detection of Prostate Cancer using Vision Transformers. In: 2022 International Conference on Artificial Intelligence in Information and Communication (ICAIIC). DOI: https://doi.org/10.1109/ICAIIC54071.2022.9722635
Jiang, X., and XU, C., 2022. Deep learning and machine learning with grid search to predict later occurrence of breast cancer metastasis using clinical data. Journal Clinical Medicine, 11, p.5772. DOI: https://doi.org/10.3390/jcm11195772
Jin, C., Luo, L., Lin, H., Hou, J., and Chen, H., 2025. HMIL: Hierarchical multi-instance learning for fine-grained whole slide image classification. IEEE Transactions on Medical Imaging, 44, pp.1796-1808. DOI: https://doi.org/10.1109/TMI.2024.3520602
Keyl, P., Bockmayr, M., Heim, D., Dernbach, G., Montavon, G., Muller, K.R., and Klauschen, F., 2022. Patient-level proteomic network prediction by explainable artificial intelligence. NPJ Precision Oncology, 6, p.35. DOI: https://doi.org/10.1038/s41698-022-00278-4
Loh, H.W., Ooi, C.P., Seoni, S., Barua, P.D., Molinari, F., and Acharya, U.R., 2022. Application of explainable artificial intelligence for healthcare: A systematic review of the last decade (2011-2022). Computer Methods and Programs in Biomedicine, 226, p.107161. DOI: https://doi.org/10.1016/j.cmpb.2022.107161
Madani, M., Behzadi, M.M., and Nabavi, S., 2022. The role of deep learning in advancing breast cancer detection using different imaging modalities: A systematic review. Cancers (Basel), 14, p.5334. DOI: https://doi.org/10.3390/cancers14215334
Qu, L., Liu, S., Liu, X., Wang, M., and Song, Z., 2022. Towards label-efficient automatic diagnosis and analysis: A comprehensive survey of advanced deep learning-based weakly-supervised, semi-supervised and self-supervised techniques in histopathological image analysis. Physics Medicine Biology, 67, p.20TR01. DOI: https://doi.org/10.1088/1361-6560/ac910a
Ramirez-Mena, A., Andres-Leon, E., Alvarez-Cubero, M.J., Anguita-Ruiz, A., Martinez-Gonzalez, L J., and Alcala-Fdez, J., 2023. Explainable artificial intelligence to predict and identify prostate cancer tissue by gene expression. Computer Methods and Programs in Biomedicine, 240, p.107719. DOI: https://doi.org/10.1016/j.cmpb.2023.107719
Shakarami, A., Nicolè, L., Terreran, M., Paolo Dei Tos, A., and Ghidoni, S., 2023. TCNN: A Transformer Convolutional Neural Network for artifact classification in whole slide images. Biomedical Signal Processing and Control, 84, p.104812. DOI: https://doi.org/10.1016/j.bspc.2023.104812
Shao, Z., Bian, H., Chen, Y., Wang, Y., Zhang, J., Ji, X., and Zhang, Y., 2021. TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification. Cornell University, New York.
Springenberg, M., Frommholz, A., Wenzel, M., Weicken, E., Ma, J., and Strodthoff, N., 2023. From modern CNNs to vision transformers: Assessing the performance, robustness, and classification strategies of deep learning models in histopathology. Medical Image Analysis, 87, p.102809. DOI: https://doi.org/10.1016/j.media.2023.102809
Sun, X., Li, W., Fu, B., Peng, Y., He, J., Wang, L., Yang, T., Meng, X., Li, J., Wang, J., Huang, P., and Wang, R., 2023. TGMIL: A hybrid multi-instance learning model based on the transformer and the graph attention network for whole-slide images classification of renal cell carcinoma. Computer Methods Programs Biomedicine, 242, p.107789. DOI: https://doi.org/10.1016/j.cmpb.2023.107789
Teramoto, A., Kiriyama, Y., Tsukamoto, T., Sakurai, E., Michiba, A., Imaizumi, K., Saito, K., and Fujita, H., 2021. Weakly supervised learning for classification of lung cytological images using attention-based multiple instance learning. Scientific Reports, 11, p.20317. DOI: https://doi.org/10.1038/s41598-021-99246-4
Tomczak, K., Czerwińska, P., and Wiznerowicz, M., 2015. The cancer genome atlas (TCGA): An immeasurable source of knowledge. Contemporary Oncology (Pozn), 19, pp.A68-A77. DOI: https://doi.org/10.5114/wo.2014.47136
Wani, N.A., Kumar, R., and Bedi, J., 2024. DeepXplainer: An interpretable deep learning based approach for lung cancer detection using explainable artificial intelligence. Computer Methods and Programs in Biomedicine, 243, p.107879. DOI: https://doi.org/10.1016/j.cmpb.2023.107879
Wibawa, M.S., Lo, K.W., Young, L.S., and Rajpoot, N., 2022. Multi-Scale Attention-based Multiple Instance Learning for Classification of Multi-Gigapixel Histology Images. Cornell University, New York, pp.635-647. DOI: https://doi.org/10.1007/978-3-031-25082-8_43
Zheng, T., Jiang, K., and Yao, H., 2024. Dynamic Policy-Driven Adaptive Multi Instance Learning for Whole Slide Image Classification. IEEE, Vancouver, pp.8028-8037. DOI: https://doi.org/10.1109/CVPR52733.2024.00767
Zheng, Y., Gindra, R.H., Green, E.J., Burks, E.J., Betke, M., Beane, J.E., and Kolachalama, V.B., 2022. A graph-transformer for whole slide image classification. IEEE Transactions on Medical Imaging, 41, pp.3003-3015. DOI: https://doi.org/10.1109/TMI.2022.3176598
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Bnar M. Ghafour, Abbas M. Ali
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Authors who choose to publish their work with Aro agree to the following terms:
-
Authors retain the copyright to their work and grant the journal the right of first publication. The work is simultaneously licensed under a Creative Commons Attribution License [CC BY-NC-SA 4.0]. This license allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
-
Authors have the freedom to enter into separate agreements for the non-exclusive distribution of the journal's published version of the work. This includes options such as posting it to an institutional repository or publishing it in a book, as long as proper acknowledgement is given to its initial publication in this journal.
-
Authors are encouraged to share and post their work online, including in institutional repositories or on their personal websites, both prior to and during the submission process. This practice can lead to productive exchanges and increase the visibility and citation of the published work.
By agreeing to these terms, authors acknowledge the importance of open access and the benefits it brings to the scholarly community.
Accepted 2026-03-27
Published 2026-06-04
ARO Journal is a scientific, peer-reviewed, periodical, and diamond OAJ that has no APC or ASC.