Abstract: Desertification has posed a serious threat to regional land restoration, vegetation recovery, as well as soil and water conservation in agricultural remediation. Considerable attention has been paid to desertification monitoring in northwest and southwest China. However, it is still lacking on the investigation of desertification in the red-bed hilly areas of southeast China. Multi-spectral remote sensing can be expected to monitor the areas of the red-bed desertification. It is also challenging to characterize and distinguish the complex surface cover types in the red-bed areas using only limited spectral features. This study aims to classify the red-bed desertification scenes considering the time-series statistical features of Vis-NIR spectral indices. The study area was taken as the red-bed exposure zone in the northern part of the Hunan-Jiangxi border area, China. Firstly, the sensitive spectral indices were constructed using the spectral features of different land cover types. Then the time series analysis was performed on these indices, including the Max, Min, Mean, Median, StdDev, and Variance. The optimal set of features was selected using the Jeffries-Matusita (J-M) distance. A hierarchical decision tree (DT) model was constructed for the typical surface cover scenes in the red-bed areas. The spatial constraints were used to optimize using expert knowledge, such as the object-based image analysis (OBIA), buffer zones, and mathematical morphology filtering. The confusion was reduced among bare rock, cultivated land, and building. Finally, the enhanced features were obtained to combine the spectral index features with the time series statistical features. The results indicated that the differentiation of the various land cover types was effectively improved in the red-bed desertification scenes. The J-M distances between the categories to be classified and the rest were mostly greater than 1.8 under the enhanced features. The temporal analysis of sensitive spectral indices was effectively enhanced to separate the typical cover scenes in the red-bed areas. The spatial pattern of the bare rock was also optimized to classify the hierarchical decision model. After that, the spatially optimized hierarchical decision model was improved the overall accuracy (OA) by 3.04%, 3.52%, and 2.87%, respectively, compared with the random forest (RF), support vector machine (SVM) models, and the original DT model. The extraction accuracy of the bare rock was 86.15%, with a recall rate of 89.31% and an F1 score of 0.88. The F1 score shared an improvement of 4.76% and 6.02%, respectively, compared with the RF and SVM. The overall accuracy of classification was significantly improved for this land cover type, compared with the rest. The misclassification and omission of bare rock were effectively reduced to optimize the mixed pixels among bare rock, building, and cultivated land. Ultimately, the classification accuracy was enhanced in the red-bed desertification scenes. A highly precise and scalable remote sensing classification can be expected to monitor the land degradation. The finding can offer a simple, effective technical solution to the desertification remote sensing. The reliable spatial data can deliver for the land restoration and agricultural remediation in the red-bed desertification areas.