DoubleAANet: Auxiliary attention and area adaptive loss for robust polyp segmentation

Colorectal cancer (CRC) remains one of the most prevalent and life-threatening diseases globally. In the development of computer-aided diagnosis systems for CRC, accurate polyp segmentation plays a pivotal role. Despite technological advancements, the segmentation of diminutive and multiple polyps continues to remain significant challenges. To address these issues, we introduce an auxiliary attention module (AAM), which leverages a channel attention module, a spatial attention module, and a feature enhancement module to simultaneously capture both global and local features. This parallel learning approach enhances the ability of model to focus on global details and localized regions, particularly for small and flat polyps. Additionally, we propose an area adaptive loss (AAL) to mitigate the under-segmentation of diminutive and multiple polyps. The AAL dynamically adjusts loss weights based on the size and quantity of polyps, prioritizing smaller and more hard samples during training. This adaptive mechanism ensures that the model is better equipped to handle imbalanced and difficult cases, thereby improving segmentation accuracy. Our proposed framework, DoubleAANet, integrates the AAM and AAL to emphasize difficult examples and localized information, enhancing the robustness and precision. To validate the effectiveness and generalization capability of our approach, we conduct extensive experiments on three diverse datasets—Kvasir-SEG, CVC-ClinicDB, and Kvasir-Sessile—as well as a cross-dataset evaluation. The results demonstrate state-of-the-art performance, with our method achieving 86.65% mIoU and 92.17% mDSC on Kvasir-SEG, 90.00% mIoU and 94.58% mDSC on CVC-ClinicDB, and 77.22% mIoU and 82.39% mDSC on Kvasir-Sessile. Notably, on the Kvasir-Sessile dataset, which contains small, flat, and diminutive polyps, our approach achieves a 12.17% improvement in recall, effectively addressing the issue of missed polyps while maintaining a superior balance between recall and precision. The cross-dataset evaluation further underscores the generalization ability of DoubleAANet, achieving 76.90% mIoU and 84.77% mDSC. Comprehensive experimental results confirm that our proposed method outperforms most existing state-of-the-art segmentation techniques, offering a significant advancement in polyp segmentation for CRC diagnosis. Source code will be available at https://github.com/feixiangdu/DoubleAANet.