Abstract: Accurate segmentation of key parts in chicken carcass is crucial for intelligent cutting systems in modern poultry processing industry. To address the issues of false detection, missed detection, and inaccurate segmentation of key parts of chicken carcasses in complex industrial scenarios (e.g., adhesion between wings and drumsticks, occlusion, and uneven lighting), this study aimed to develop a lightweight, high-precision, and real-time instance segmentation model suitable for deployment on intelligent chicken carcass cutting equipment. First, an enhanced chicken carcass dataset was constructed, focusing on Sanhuang chickens and white-feathered chickens. It was expanded from 1090 original images to 5450 images, covering three types of scenarios: changes in ambient lighting, carcass occlusion, and compression-induced deformation. Multi-dimensional data augmentation techniques such as geometric transformation, illumination adjustment, and occlusion simulation were adopted to improve the model’s robustness. Second, the DEF-YOLO-seg model was developed by improving the YOLOv12n-seg as the baseline: (1) The C3k2_DAttention module was designed by fusing the C3k2 module with Deformable Attention (DAttention), which replaced the Area-Attention Enhanced Cross-Feature (A2C2f) module in the lower layer of the backbone network to enhance the feature extraction capability for adhered/occluded regions; (2) The Efficient Up-Convolution Block (EUCB) was introduced to replace the Upsample module in the neck network, reducing computational cost while improving feature fusion efficiency; (3) A composite loss function (Focaler-CIoU) combining Focaler-IoU and CIoU was constructed to adapt to the distribution characteristics of easy and difficult samples in complex scenarios. Finally, model training and testing were completed on a hardware platform equipped with an NVIDIA RTX 3090 GPU and an Intel Xeon Platinum 8362 CPU. The DEF-YOLO-seg model achieved a mean Average Precision at an IoU threshold of 0.5 (mAP₅₀) of 95.5% and a mean Average Precision at IoU thresholds from 0.5 to 0.95 (mAP_50-95) of 94.1%, which were 1.3 and 2.8 percentage points higher than those of the baseline YOLOv12n-seg, respectively. With a parameter count of 3.3M and a computational complexity of 11GFLOPs, the model’s inference time per image on a local computer was no more than 30 ms. Compared with mainstream models such as YOLOv9c-seg, YOLOv11n-seg, and YOLOv12n-seg, the proposed model maintained lightweight characteristics while achieving superior segmentation accuracy. Furthermore, the parameter sensitivity analysis reveals that the optimal Focaler-CIoU configuration (d=0.22, u=0.73) precisely matches the IoU distribution characteristics of chicken carcass data, and this finding highlights the importance of task-specific loss function design rather than using generic settings. In practical production line, the model’s image-level accuracy 95%, and the Dice coefficients of the neck, wings, and drumsticks increased from 0.85, 0.83 and 0.78 to 0.93, 0.92 and 0.90, respectively. It effectively solved the problems of missed detection, false detection, and false segmentation of small parts (e.g., neck and shank) under adhesion and occlusion conditions. The DEF-YOLO-seg model achieves a balance between segmentation accuracy, real-time performance, and deployment feasibility, and can be applied to intelligent chicken carcass cutting equipment, providing technical support for the intelligent upgrading of the food processing industry. Future research will focus on developing cutting path planning technology based on this model and further optimizing the balance between lightweight design and detection accuracy.