Method for detecting rice flowering spikelets using visible light images

Rice flowering spikelets bloom generally at 10:00-12:00, especially when the temperature is 24-35 ℃ and the relative humidity is 70%-90%. Therefore, the flowering time is necessary to be accurately determined for the timely pollination in the production of hybrid rice seed. In this study, the images were captured by a visible light camera at two flowering characteristics, including the opening of spikelet hull, and the emesis of spikelet anthers. Series Otsu (SOtsu) was applied in tandem to extract the spikelet anthers through the visible light blue channel. An attempt was made to detect the flowering status of rice glumes using visible images, in order to meet the needs of hybrid rice seed pollination. A Canon single-lens reflex (SLR) camera was adopted for data acquisition, which was a benefit to segment the image using the tandem SOtsu. Deep learning models, such as FasterRCNN and YOLO-v3, were used to identify the spikelet anthers and the opening spikelet hull. The most suitable method was selected for flowering characteristics detection to compare the precision, recall, and the F1 coefficient of different models. Two datasets of visible light images were set for spikelets (15 cm and 45 cm imaging distance), each of which used two characteristics. A labeling software was applied to label the category and position of images. As such, a sample database was established for the training of detection models with deep learning. The performance of three models, including SOtsu, FasterRCNN, and YOLO-v3, were evaluated, where the detection was verified from multiple angles. The experiment was also conducted for the model robustness as well. The maximum inter-class variance was utilized in the SOtsu to separate the foreground (rice) from the background using the grayscale image of B-channel, where the grayscale of the background was set to be zero. An analysis was then made for the maximum inter-class variance that applied independently in the pixel range of extracted region, and then the spikelet anthers were further separated from the spikelets hull. The original gray values of spikelet anthers were retained, while the gray values of spikelet hull were set to be zero. Finally, the extraction was evaluated to combine with original images and the number of connected areas that were calculated by the eight-connected output as well. The results showed that the precision, recall rate, F1 coefficient and Pearson correlation coefficient of FasterRCNN model in spikelet hull detection were 1, 0.97, 0.98, and 0.993, respectively, while those of SOtsu in spikelet anthers detection were 0.92, 0.93, 0.93, and 0.936, respectively. It inferred that the SOtsu and FasterRCNN models were both capable of rice flowering detection, but the opening spikelet hull was more suitable than the spikelet anthers for the rice flowering features detection with deep learning model. The model robustness indicated that the highest stability was achieved in the FasterRCNN model to identify the spikelet flowering status with high precision under low, high and uneven light conditions. In addition, the spikelet anthers that opened on the same day split and pollened in 3-5 min, and withered on the same day. There was no recognition significance in the withered spikelet anthers without pollen. It was also necessary to verify the classification ability of detection for the withered anthers, in order to avoid the wrong identification of withered anthers. The SOtsu performed well in the image segmentation using the gray value for the withered spikelets anthers. The SOtsu was also better than FasterRCNN in the identification of withered spikelets. Correspondingly, the SOtsu was expected to replace the FasterRCNN model for the flowering spikelet before the completion of model construction, in order to ensure the detection continuity of rice flowering spikelet. The influencing factors of recognition were reduced to control the process of detection. Since the segmentation was processed for morphological opening operations, there was also some limitation in the recognition of overlapping anthers. A further study can be followed by a more in-depth exploration of high-throughput detection.