Abstract: An accurate and rapid identification is often required for the intertwined crops and weeds in the natural environment. However, many kinds of weeds cannot be identified accurately in real time. In this study, weed identification was proposed using the AAF-DeepLabv3 model. The potato seedlings and their associated weeds were also taken as the research objects. Firstly, the backbone network was replaced with the MobileNetV2, according to the DeepLabv3 semantic segmentation. A lightweight DeepLabv3 model was established to improve the nonlinear regression of the model. An attention activation function (AAF) was proposed using an attention mechanism. The AAF-Conv convolution was also integrated to replace the lightweight DeepLabv3 in the semantic segmentation. The backbone network MobileNetV2 was the first 3×3 Conv. As such, the AAF-DeepLabv3 model was established after optimization. The AAFDeepLabv3 model was used to obtain the morphological boundaries of the potato seedlings. The weed areas were identified from the images using imaging techniques. The AAF activation function was then compared with the common ones, according to the lightweight DeepLabv3 model. The mean intersection over Union (mIoU) increased by 1.58, 1.31, and 1.99 percentage points, compared with the ReLU6, SiLU, and CeLU, respectively. While the mean pixel accuracy (mPA) increased by 1.47, 0.6, and 1.26 percentage points, respectively, indicating better performance. The AAF-DeepLabv3 model also shared significant performance advantages over the other common semantic segmentation. The mIoU and mPA of 90.82% and 95.56% were 1.07 and 1.15 percentage points higher than the original DeepLabv3 model, respectively. The frame rate was 69.21 frames/s, which was 30.77 frames/s higher than the original model. While the model size was 22.56 MB, which was 185.96 MB lower than the original model. The overall performance of the model was better than that of the mainstream network models in the semantic segmentation, such as the UNet, PSPNet, HrNet, DeepLabv3, and FCN. The semantic segmentation of the AAF-DeepLabv3 model was much more accurate to segmentate the potato seedling images. There was also the fine contour segmentation for the potato seedling image boundary. The AAF-DeepLabv3 model shared the excellent performance to accurately segment the potato seedlings with the small targets or the growing together with weeds. Finally, there was a decrease in the number of images that were annotated in the early stage after weed recognition, compared with object detection and ordinary semantic segmentation. The effective identification was also realized to reduce the overlap of the weeds and crop targets in the wide variety of weeds. This finding can provide a technical reference to develop mobile terminal equipment and intelligent weeding devices to identify the weeds in farmland.