Multi-modal Brain Tumour Segmentation Using Transformer with Optimal Patch Size

Abstract

Early diagnosis and grading of gliomas are crucial for determining therapy and prognosis. For this purpose, magnetic resonance (MR) studies of brain tumours are commonly used. Due to the overlap between the intensity distributions of healthy, enhanced, non-enhancing, and edematous areas, automated segmentation is difficult. Convolutional neural networks (CNNs) have been utilized as the dominant networks for segmentation tasks. However, they suffer from the inability to capture long-range dependencies and global features due to their limited kernels. Vision transformers (ViTs) were introduced recently to tackle these limitations. Although transformers are capable of capturing long-range features, their segmentation performance falls as the variety of tumor sizes increases. In this paper, we propose a framework to find the ViT optimal patch resolution for the brain tumor segmentation task, particularly for segmenting smaller tumors. We validated our proposed framework on the BRaTS'21 dataset. This research lays the groundwork for future research on the semantic segmentation of tumor segmentation and detection using vision transformer-based networks for optimal outcomes.