Automated segmentation of wisdom teeth (m3): an analytical study of model performance & metric variability

Abstract

Segmentation of wisdom teeth (M3) from panoramic dental radiographs plays a critical role in dental diagnosis, treatment planning, and surgical decision-making. However, manual segmentation is time-consuming, relies heavily on expert knowledge, and is prone to inter-observer variability. Although deep learning has shown strong performance across the medical imaging domain, relatively little research has specifically addressed wisdom tooth segmentation using deep learning-based models. This study addresses that gap by conducting a comparative analysis of four semantic segmentation models: U-Net, U-Net++, DeepLabV3, and DeepLabV3+. A custom dataset of 700 panoramic dental radiographs was prepared, with pixel-level segmentation masks manually created for six semantic classes: general teeth and the four wisdom teeth defined by the FDI tooth numbering system. All models were trained and evaluated under identical conditions, including the same preprocessing, augmentation, and training setup, using a ResNet34 encoder within the PyTorch framework. Model performance was assessed using Accuracy, Precision, Recall, F1-score, Dice Similarity Coefficient (DSC), and Intersection over Union (IoU). In addition, the study examined the effects of manual versus SAM-assisted annotation and compared two dataset preparation strategies: Split → Augmentation and Augmentation → Split. Test results show that DeepLabV3 achieved the best overall performance, with the highest Dice Score (0.7530), IoU (0.6872), Precision (0.7056), and Accuracy (0.9918), while U-Net achieved the highest Recall (0.9671). Manually annotated masks consistently produced more accurate segmentation results than SAM-assisted annotations. Furthermore, applying data augmentation before splitting the dataset was found to introduce data leakage, yielding artificially inflated performance, whereas splitting the dataset before augmentation produced more reliable and realistic results. These findings suggest that the accuracy of wisdom tooth segmentation depends not only on the choice of deep learning architecture but also on annotation quality and dataset preparation strategy. This work underscores the value of a manually annotated wisdom tooth dataset, provides a comparative benchmark across four segmentation models, and offers direction for future research in automated dental image analysis.