Abstract: Sustainable agriculture can rely mainly on four dimensions at present, including the resource adaptability, economic effectiveness, ecological sustainability, and social development viability. However, it is a high demand to revert the non-grain cultivated (NGC) lands in the mountainous regions of southern China. Existing agricultural production can be disrupted to trigger the severe secondary land abandonment, particularly after the plantation destruction and reconversion. It is urgently needed to implement the contextually adaptive strategies under complex circumstances. Temporal assessments can also be incorporated for the spatially differentiated restoration. Such measures should be systematically organized, according to the diagnostic evaluations of the local socio-ecological conditions and production system resilience. This study aims to systematically analyze the criteria of the cultivated land identification in the Big Food View from the food system perspectives. The land types were identified after NGC reversion. Taking Jianyang District as the study area, the interpretation of the high-resolution remote sensing was employed to map the NGC patterns and distribution. A multi-dimensional diagnostic index system was developed to assess the cultivated land use potential. The multi-dimensional coupling coordination was also found in the NGC utilization. Finally, a self-organizing map network was applied to identify the functional constraints and then prioritize restoration sequences. The results showed that: (1) The types of primary crops included the forest land (60.08%), tea plantations (19.45%), citrus orchards (9.86%), pond aquaculture (6.27%), grape cultivation (4.15%), and dragon fruit (minimal) in the non-grain cultivated plots. These plots were predominantly distributed in the low-elevation hills (100–400 m) with the sunny slopes (6–20°), indicating the strong topographic adaptability. (2) A preliminary level was found in the multi-dimensional coupling coordination degree (MCD) of the cultivated land use, with an average MCD of 0.652. The largest proportion was observed in the primary coordination zones (46.99%), which was mainly distributed in Huangkeng, Mashan, and Xushi towns. Marginal coordination zones (35.29%) were concentrated in the east and northwest, while the high-quality and good coordination zones (14.96%) were primarily located in river valley lowlands. Intermediate coordination zones (2.66%) were clustered in southern Jianyang. While the near-dysfunctional zones (0.10%) were largely influenced by the terrain and ecological constraints. (3) The single cropland was covered 1133.68 ha in the functional deficiencies, where the social development was the most prominent limitation; The areas with two and three weaknesses were measured as 3 097.52 and 1 915.27 ha, respectively, with the resource adaptability and social development as the dominant combinations. According to the potential and deficiencies, the restoration zones were classified as the priority (30.52%), sub-priority (42.87%), medium-term (26.50%), and long-term (0.1%). In conclusion, the differentiated and precise restoration should be adopted in response to the functional limitations. Particularly, there are the dual constraints of resource adaptability and social development. Economic and ecological objectives can be balanced to effectively promote the sustainable utilization of cropland. These findings can offer a scientific basis for the targeted re-cultivation and sustainable management of non-grain cropland in southern mountainous. The theoretical and practical significance can also provide to advance the multidimensional and coordinated agriculture.