高级检索+

基于SAM分割的交互式局部图像风格迁移方法研究

Research on Interactive Local Image Style Transfer Based on SAM Segmentation

  • 摘要: 针对目前局部图像风格迁移目标区域选择困难、迁移灵活性不足、容易出现内容泄露、前景与背景边缘过渡不自然等问题,提出一种基于任意分割模型(Segment Anything Model, SAM)的交互式局部图像风格迁移方法。首先利用SAM分割网络在用户输入提示的指导下对输入的内容图像进行交互式目标迁移区域提取,对得到的有效对象掩码进行二值化处理,以二值化掩码提取全局风格化图像的目标区域作为前景、内容图像作为背景图像进行泊松融合,实现局部图像风格迁移。为了避免迁移过程中的内容泄露,全局风格迁移网络采用生成对抗网络架构,通过多级自适应注意力归一化模块进行风格特征转换,利用联合损失函数对网络进行综合训练。实验结果表明,设计的交互式局部图像风格迁移网络能够根据用户提示生成灵活可控的局部迁移结果,可以对图像中的任意物体进行风格迁移,迁移结果很好地保留了内容源图像中的内容结构,避免了内容泄露,且前景与背景边缘过渡更加自然。

     

    Abstract: Aiming at the problems of the current local image style transfer methods, such as the difficulty in selecting the target region, the lack of transfer flexibility, the easy occurrence of content leakage and unnatural transitions between foreground and background boundaries, an interactive local image style transfer method based on SAM segmentation is proposed in this paper. Firstly, SAM segmentation network is employed to extract the target transfer region of the input content image interactively under the guidance of the user input prompt, and the effective object mask is binarized. The binary mask is used to extract the target region of the global stylized image as the foreground and the content image as the background image for Poisson fusion to realize the local image style transfer. In order to avoid content leakage during the transfer process, the architecture of generative adversarial network is adopted in the global style transfer network. The multi-level adaptive attention normalization module is used for style feature conversion, and the joint loss function is used for comprehensive training of the network. The experimental results show that the interactive local image style transfer network designed in this paper can generate flexible and controllable local transfer results according to user prompts, and can carry out style transfer for any object in the image. The transfer results well preserve the content structure in the content source image, avoid content leakage and the boundaries of foreground and background transit more naturally.

     

/

返回文章
返回