高级检索+

基于改进GAN的银杏叶用林遥感图像超分辨重建模型

A super resolution reconstruction model for drone photography of Ginkgo forest based on generative adversarial networks

  • 摘要: 为解决无人机低空遥感多光谱成像中飞行作业效率与图像分辨率难以兼顾、进而影响作物生长情势监测精度的问题,该研究设计了一种图像超分辨重建模型-残差变换生成对抗网络(residual transformer generative adversarial network, RTGAN)。方法上,构建银杏叶用林冠层多光谱真实高 / 低分辨率图像数据集,引入多重密集残差块(multiple dense residual block, MDRB),融合U-Net和Transformer模块优化网络结构,在银杏叶用林样地验证 RTGAN 模型性能。结果显示,利用 RTGAN重建银杏林低分辨率遥感图像后,银杏冠层纹理清晰,超分辨率后图像的峰值信噪比(peak signal-to-noise ratio,PSNR)、结构相似度(structural similarity index measure,SSIM)分别平均提升了67.22%、74.54%;感知相似度指标(learned perceptual image patch similarity, LPS)、Fréchet inception距离(Fréchet inception distance, FID)分别平均缩小了84.42%和90.50%;银杏叶产量估测精度的相关系数r平均提升33.34%,接近较低飞行高度采集的高分辨率图像估测精度(r=0.83)。本文提出的RTGAN超分辨率重建技术可提升低分辨率遥感图像的银杏叶产量估测模型精度、无人机作业效率及遥感图像抗环境干扰能力,为银杏林智慧种植提供技术支撑。

     

    Abstract: The growth status of crops can be effectively monitored using unmanned aerial vehicle (UAV) multispectral images. However, monitoring accuracy is influenced by the resolution and clarity of remote sensing images. While increasing the UAV flight altitude improves flight efficiency, it simultaneously reduces image resolution, which significantly impairs monitoring accuracy. To address this challenge, this study proposes a novel image super resolution (SR) reconstruction model, named residual transformer generative adversarial network (RTGAN), designed to effectively balance flight efficiency and monitoring accuracy. First, a multispectral image dataset of ginkgo canopies was constructed, comprising both high resolution (HR) and low resolution (LR) remote sensing images. HR images were captured by UAV at an altitude of 15 m. The LR image dataset consists of LR30 and LR60, captured at altitudes of 30 and 60 m, respectively. These raw images underwent a series of multispectral image preprocessing procedures, including image stitching, radiometric calibration, multi-channel integration, image registration, and image cropping. The number of preprocessed images reached 10,000, forming the real HR/LR image datasets of ginkgo canopies, which were used to train the RTGAN model. Next, this study improved the network loss function by incorporating pixel loss, adversarial loss, perceptual loss, and regularization loss. The SR network architecture comprised a generator network and a discriminator network. The generator network was optimized by introducing multiple dense residual block (MDRB) to extract global features from remote sensing images, while the discriminator network integrated a U-Net module and a Transformer module. These enhancements improved RTGAN's ability to process complex textures and strengthened its capacity to generate high-quality SR images. Finally, the usability and validity of the RTGAN model were evaluated by assessing the accuracy of ginkgo leaf yield prediction. Correlation analysis of vegetation indices was performed to select suitable indices. Multiple linear regression (MLR), partial least squares regression (PLSR), and random forest regression (RFR) models were employed to establish yield prediction models using HR, LR, and SR images. The comparison results showed that the real HR/LR image dataset could reveal detailed textures and structural features across images of different resolutions, enhancing the SR model's reconstruction capability and generalization performance. The ginkgo leaf yield prediction performance before and after SR was compared, revealing that the texture of ginkgo canopies was significantly clearer after SR by RTGAN. For the SR images, the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) increased by an average of 67.22% and 74.54%, respectively; learned perceptual image patch similarity (LPIPS) and Fréchet inception distance (FID) decreased by an average of 84.42% and 90.50%, respectively; and the correlation coefficient (r) for ginkgo leaf yield estimation accuracy improved by 33.34%, approaching the yield estimation accuracy of HR images collected at lower flight altitude (r = 0.83). Therefore, the RTGAN model for SR technology can effectively enhance the accuracy of ginkgo yield prediction models derived from LR images while maintaining high flight efficiency.In summary, RTGAN enhances the robustness of remote sensing images against environmental interference and addresses the practical demands of large-scale monitoring. It holds significant potential for application and research in the smart cultivation of ginkgo.

     

/

返回文章
返回