Enhanced Cancer Diseases Prediction Using Vision Transformer (ViT) and Multiple Transfer Learning Based on CNN Algorithm
DOI:
https://doi.org/10.69055/jcsir.v1i1.5Keywords:
Cancer Detection, Transfer Learning, Vision Transformer (ViT), Lung Cancer, Brain Tumor, Medical Image ClassificationAbstract
Lung and brain cancers remain among the leading causes of mortality worldwide, with early detection being critical for improving patient outcomes. Deep learning, particularly Convolutional Neural Networks (CNN) enhanced through transfer learning (TL), has demonstrated remarkable potential in medical image classification tasks. This study proposes two complementary approaches: (1) a multi-stage CNN transfer learning framework trained sequentially on public brain tumor MRI images (7,022), public lung cancer CT images (1,190), and a private clinical dataset (500 images); and (2) an enhanced methodology replacing the CNN backbone with a Vision Transformer (ViT-B/16) pretrained on ImageNet-21k for superior global feature extraction. Both models underwent rigorous preprocessing including image resizing, Gaussian blurring, and Min-Max normalization. The CNN-based transfer learning model achieved accuracy of 92.81% (Scratch), 97.21% (Pre-Trained), and 99.40% (Transfer Pre-Pre-Trained) on successive datasets. The enhanced ViT model further improved performance, achieving 98.34%, 98.67%, and 99.81% on brain, lung, and private datasets respectively. Both approaches significantly outperform prior methods. The ViT-enhanced model demonstrates the superior generalization capability of Transformer architectures for multi-organ cancer classification from heterogeneous medical imaging datasets.
References
[1] P. A. Dutta, S. K., and M. Dutta, “Multi-classification of Brain Tumor Images Using Transfer Learning Based Deep Neural Network,” in International Conference on Artificial Intelligence for Smart Community, 2020, pp. 927–933. DOI: https://doi.org/10.1007/978-981-16-2183-3_87
[2] K. T. Chui, G. B. Bandara, J. R. Hina, et al., “Multiround transfer learning and modified generative adversarial network for lung cancer detection,” International Journal of Intelligent Systems, vol. 2023, pp. 1–14, 2023. DOI: https://doi.org/10.1155/2023/6376275
[3] S. J. Pan and Q. Yang, “A Survey on Transfer Learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345–1359, 2010. DOI: https://doi.org/10.1109/TKDE.2009.191
[4] K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning,” Journal of Big Data, vol. 3, no. 1, p. 9, 2016. DOI: https://doi.org/10.1186/s40537-016-0043-6
[5] F. Zhuang, Z. Qi, K. Duan, et al., “A Comprehensive Survey on Transfer Learning,” Proceedings of the IEEE, vol. 109, no. 1, pp. 43–76, 2021. DOI: https://doi.org/10.1109/JPROC.2020.3004555
[6] S. Bozinovski, “Reminder of the first paper on transfer learning in neural networks, 1976,” Informatica, vol. 44, no. 3, pp. 291–302, 2020, doi: 10.31449/inf.v44i3.2828. DOI: https://doi.org/10.31449/inf.v44i3.2828
[7] B. Qiao, J. Kang, A. Jiao, et al., “A novel transfer-learning based physician-level general and subtype classifier for non-small cell lung cancer,” Heliyon, 2022. DOI: https://doi.org/10.1016/j.heliyon.2022.e11981
[8] M. Humayun, R. Sujatha, S. N. Almuayqil, and N. Z. Jhanjhi, “A Transfer Learning Approach with a Convolutional Neural Network for the Classification of Lung Carcinoma,” Healthcare (Basel), vol. 10, p. 1058, 2022. DOI: https://doi.org/10.3390/healthcare10061058
[9] K. Lakshmi, T. M. Prabha, and D. Anjali, “Convolutional neural network in the detection of lung carcinoma using transfer learning approach,” International Journal of Imaging Systems and Technology, vol. 30, no. 2, pp. 445–454, 2020. DOI: https://doi.org/10.1002/ima.22394
[10] S. Deepak and P. M. Ameer, “Brain tumor classification using deep CNN features via transfer learning,” Computers in Biology and Medicine, vol. 111, p. 103345, 2019. DOI: https://doi.org/10.1016/j.compbiomed.2019.103345
[11] S. V. Modiya and V. Priyanka, “Brain Tumor Detection Using Transfer Learning with Dimensionality,” International Journal, vol. 10, no. 2, pp. 201–206, 2022.
[12] S. Hossain, A. Chakrabarty, T. R. Gadekallu, M. Alazab, and M. J. Piran, “Vision Transformers, Ensemble Model, and Transfer Learning Leveraging Explainable AI for Brain Tumor Detection and Classification,” IEEE Journal of Biomedical and Health Informatics, 2023. DOI: https://doi.org/10.1109/JBHI.2023.3266614
[13] M. I. Mahmud, M. Mamun, and A. Abdelgawad, “A deep analysis of transfer learning based breast cancer detection using histopathology images,” in Proc. 10th Int. Conf. Signal Processing and Integrated Networks (SPIN), 2023, doi: 10.1109/SPIN57001.2023.10117110. DOI: https://doi.org/10.1109/SPIN57001.2023.10117110
[14] S. Asif, M. Zhao, F. Tang, and Y. Zhu, “An enhanced deep learning method for multi-class brain tumor classification using deep transfer learning,” Multimedia Tools and Applications, pp. 1–28, 2023. DOI: https://doi.org/10.1007/s11042-023-14828-w
Downloads
Published
Issue
Section
License
Copyright (c) 2025 The Author(s)

This work is licensed under a Creative Commons Attribution 4.0 International License.