Advances in plant phenology observation and prediction driven by modern imaging technologies and machine learning

Plant phenology aims to explore the interactions between plant growth cycles and the natural environment. It is of great significance to acquire plant phenological information and processing. The plant growth status can be accurately predicted to select and then breed the high-quality cultivars. However, manual observation cannot fully meet the large-scale production in recent years, due to the time-consuming, labor-intensive, subjectivity and reporting delays. The phenological observation platforms can be expected to acquire high-throughput and multi-scale data using advanced imaging sensor technologies. Particularly, machine learning can then be employed to establish the corresponding phenological models. The key time points can also be recognized for the precise prediction of the plant phenological stages. This review introduced the manual phenological and emerging sensor observation. A systematic analysis was made on the monitoring and data acquisition of the plant phenology, particularly in the applications of proximal sensing systems, phenological flux towers, unmanned aerial vehicle (UAV) platforms, and satellite remote sensing platforms. Additionally, two categories of the prediction models were summarized for the plant phenology: one was the environmental factor-driven models using mechanistic understanding, and another consisted of the data-driven models that relied on statistical analysis and machine learning. Furthermore, a systematic review was also given on the machine learning technologies for modeling phenology at different biological scales, including intra-species, inter-species, and community-level phenology. An emphasis was also placed on the practical value of phenological prediction in different fields, such as plant cultivation, growth dynamics monitoring, and crop yield forecasting. Current research indicated that the imaging technologies were integrated with advanced machine learning, in order to promote the automation of the plant phenological observation for the high accuracy of phenological predictions. Nevertheless, there were major challenges in the application of imaging sensors and machine learning on plant phenology. Three aspects were then summarized on these challenges: Firstly, the precision of the phenological feature extraction was confined to the spatial and temporal resolution of the static observation scales and single-sensor modalities; Secondly, the multi-source data fusion was prone to induce the dimensionality for the high computational complexity of the feature space; And thirdly, the existing phenological models generally exhibited the strong scale dependence, leading to the adaptive applicability across multiple scales from the organ to individual, population, and ecosystem levels. Looking forward, plant phenology can be expected to focus mainly on the continuous collection and monitoring of multi-scale, long-term phenological datasets, as well as the integration and fusion of multimodal imaging sensor data. The collaborative system of phenological observation can also be developed to synergize the spaceborne, airborne, and ground-based platforms. Moreover, there is an urgent need to develop phenological prediction models with self-learning, self-optimizing, and generalization. The AI models can be applied to fuse the heterogeneous observation data from the different platforms. The cross-platform model can be calibrated and validated to accelerate the automation, digitization, and informatization of plant phenological monitoring. These findings can also provide more precise and efficient technological support for agricultural production and ecological protection against climate change.