Abstract: Real-time monitoring of the grain breakage and impurity rates is often required during rice harvesting in modern agriculture. In this study, an intelligent monitoring system was proposed to integrate a stratified sampling device with a deep learning model. An accurate, non-destructive, and online detection was achieved in the rice harvest quality. Concurrently, the physical structure was also enhanced for image acquisition. The detection algorithm was improved for the densely distributed and small targets. In the hardware component, a three-stage intermittent sampling was composed of a grooved wheel, porous deflector plates, and adjustable baffles. The layered dispersion of rice grains was also facilitated to suppress the particle overlap. The structure was optimized using Discrete Element method (DEM) simulations in EDEM software. The optimal combination of the parameters was achieved with a baffle gap of 7.5 mm, deflector angle of 50°, and conveyor speed of 0.15 m/s, particularly with the lowest particle distribution coefficient of variation (CV = 0.152). At the same time, the average occlusion rate remained at a moderate level. Grain stratification was evaluated to rely mainly on the coupled CV and occlusion rate, in order to balance the distribution uniformity and particle overlap. There was the superior uniformity and flowability of the grain spread. On the software side, the YOLOv8n network was enhanced for the embedded application. Shuffle Attention (SA) modules were incorporated to capture the dense and small-object features. While the VoV-GSCSP module reduced the parameter count and complex computation. The conventional CIoU loss was replaced by the PIoU2 loss function. The localization errors in the overlapping targets were used to incorporate the normalized positional deviation and geometric penalty terms. Ablation studies demonstrated that the performance of these components: The improved model was achieved in a mean average precision (mAP@50) of 95.3%, F1-score of 92.5%, and a real-time inference speed of 126 frames/s with only 2.9 million parameters. Field and bench-scale experiments were conducted using the “Lingliangyou-268” rice variety. In controlled bench trials, the predicted average error rates were 3.50% and 6.27%, respectively, for the grain breakage and impurity. Field tests were performed on the 4LZ-3.2 combine harvester equipped with a Jetson Orin Nano-based embedded system. The robust and real-time performance was achieved in the average relative errors of 5.52% for the breakage and 6.77% for the impurity under varying operating speeds. The high accuracy and stability were also realized under complex agricultural conditions. A cost-effective, lightweight, and highly accurate rice quality monitoring solution was suitable for deployment in the intelligent agricultural machinery. The hardware and software optimization can also provide a reliable framework for the real-time field deployment during crop harvesting in precision agriculture.