Abstract: As a distinctive specialty agricultural product, fresh daylily (Hemerocallis citrina) is not only valued for its nutritional properties but also plays a significant role in the agricultural economy of several regions. The post-harvest quality grading of fresh daylily is a critical step in its commercial processing, storage, and subsequent deep utilization, directly influencing both market value and processing efficiency. However, current grading practices predominantly rely on manual sorting, which is characterized by low efficiency, high labor intensity, and strong subjectivity, resulting in inconsistent grading outcomes. Moreover, existing automated detection models often suffer from high computational complexity and insufficient recognition accuracy, particularly when dealing with dense arrangements and morphological variations inherent to fresh daylily buds. To address these challenges, this study establishes a systematic three-level grading framework based on bud length as the primary criterion. A pixel threshold conversion model was developed through spatial calibration and geometric correction, enabling the accurate mapping of physical bud lengths to image pixel thresholds. This conversion supports precise image-based grading annotation and lays the foundation for automated visual inspection. Building upon this framework, a novel quality grading detection model, named PDi-YOLOv11n, is proposed based on an improved YOLOv11n architecture specifically tailored for fresh daylily. First, to enhance feature extraction under dense and complex arrangement scenarios while reducing computational redundancy, the conventional C3k2 module in the backbone network was replaced with a C3k2_CSPPC module, which integrates a Cross-Stage Partial Pyramid Compression (CSPPC) structure. This module effectively refines multi-scale feature representations by compressing spatial and channel information across hierarchical stages, thereby improving the model’s ability to discriminate overlapping or closely adjacent daylily buds. Second, to address the persistent issues of missed detection and inaccurate localization—especially for buds with varying orientations and occlusions—a dynamic upsampling module (DySample) was incorporated. Unlike fixed upsampling methods, DySample employs a content-adaptive mechanism that dynamically adjusts sampling locations based on feature context, significantly enhancing spatial localization accuracy and enabling more precise boundary delineation. Third, an iEMA mechanism was introduced into the neck network, integrating the Inverted Residual Mobile Block (iRMB) with the Efficient Multi-scale Attention (EMA) mechanism. This integration facilitates the extraction of salient features of fresh daylily by emphasizing discriminative regions across multiple scales while effectively suppressing background interference. The combined architecture thus improves detection efficiency and robustness under varying field conditions. Experimental evaluations were conducted on a self-constructed dataset comprising diverse images of fresh daylily collected under different lighting, orientation, and density conditions. Compared with the baseline YOLOv11n model, the proposed PDi-YOLOv11n achieved substantial improvements in detection precision across all grading categories. Specifically, the average detection precision for premium, first-grade, and second-grade fresh daylily increased by 2.6%, 5.3%, and 2.6%, respectively. Overall model performance showed notable gains, with precision, recall, and mean average precision improving by 4.3, 2.4, and 2.6 percentage points, respectively. In addition to accuracy improvements, model lightweighting was effectively achieved: the floating-point operations, number of parameters, and model size were reduced by 3.3%, 12.8%, and 13.0%, respectively, compared to the original model. These reductions demonstrate that the proposed modifications not only enhance detection capability but also improve computational efficiency, making the model suitable for resource-constrained deployment scenarios. To assess practical applicability, the model was deployed in an Android-based mobile detection system. Under real-world testing conditions using high-resolution images, the system maintained high grading detection accuracy while achieving an average inference time of 0.98 seconds per image on mobile devices. This demonstrates the model’s capability to support real-time or near-real-time field applications. Overall, the PDi-YOLOv11n model achieves a favorable balance between detection accuracy, computational efficiency, and deployment feasibility. The integrated approach—combining a robust length-based grading standard, targeted architectural enhancements, and effective lightweighting—provides a reliable and scalable solution for automated fresh daylily grading, offering substantial technical support for advancing post-harvest processing and quality control in the specialty agricultural sector.