Abstract: Soil erosion has posed a substantial threat to the ecological stability of sustainable agriculture. Check dams can be expected to constitute the vital infrastructure for soil and water conservation in the Loess Plateau. It is often required to precisely delineate the check dam spatial extents with accurate point positioning. Among them, the Malian River Basin on the Loess Plateau is one of the most typical representatives of severe soil erosion and large-scale construction of the check dams. The heterogeneous distribution patterns and diverse land cover can be expected to provide a large number of samples for the extraction of the check dams. However, the conventional extraction can be limited to the fuzzy boundaries in the imagery and spectral confusion with the terraced fields. Manual interpretation can also hinder the precise identification of the areas and points of the check dam. In this study, a semantic segmentation model was proposed for the check dam extraction using object-based image analysis (OBIA) and the U-Net model, termed as UNet-OBIA. A multi-source remote sensing image dataset of the check dams (RGB+DEM) was constructed, including the complex background information from the four types of features: terraced fields, vegetation, bare land, and buildings. The scale parameter was selected for the optimal segmentation after estimation. The U-Net network initially extracts the potential areas of the check dam. Subsequently, the object image analysis was applied for the multi-scale segmentation of the check dam boundaries. The majority voting was employed for the feature fusion. Then, the check dam areas were optimized to combine the terrain parameters, such as the elevation. Finally, the body points of the check dam were precisely located via hydrological analysis. The UNet-OBIA was validated in the three representative areas within the Malian River Basin on the Loess Plateau. A comparison was conducted against the FCN, DeepLabv3+, PSPNet, and UNet++ models (all integrated with the OBIA), in order to further validate the accuracy of the approach. The results showed that the UNet-OBIA achieved significant performance with the precision (92.47%), recall (87.16%), overall accuracy (95.82%), F₁ score (89.73%), and mean intersection over union (81.41%). Compared with the baseline U-Net model, these metrics were improved by 2.83, 1.27, 1.25, 2.01, and 3.82 percentage points, respectively, indicating the optimal overall performance of the UNet-OBIA. Experimental results demonstrated that the DEM topographic data were integrated with the remote sensing imagery to construct a four-band dataset. The OBIA post-processing was combined to significantly improve both the completeness of the check dam boundary extraction and the accuracy of the dam shape recognition in complex backgrounds. Furthermore, the hydrological analysis was achieved with a high accuracy of 80.30% after point recognition, and a completeness rate of 86.88%, all of which indicated the significant robustness and generalization of the approach for the check dam extraction in the complex terrain of the Loess Plateau. This accurate and efficient extraction can provide the vital data framework to optimize the spatial layout of the check dam. Hydrological simulations can be expected to enhance the soil erosion control in the Loess Plateau watersheds, and help optimize the watershed strategies and conservation planning frameworks.