Abstract: Rural housing land is ever-increasing with the advancement of urbanization in China, despite the rapid outflow of the rural population. Intensive use of rural settlements can also boost rural revitalization under the supply of urban construction land. Taking Shanghai City as an example, an empirical study was carried out to examine the spatiotemporal pattern and driving factors of the interaction relationship between rural settlements and rural population. A series of models were combined, such as the elastic coefficient model, Tapio decoupling model, and LMDI decomposition. The results showed that: 1) There was a serious imbalance structure of “rural population decreasing and residential land increasing” since 1978. Rural settlements and population had also experienced the process of “increasing population and expanding residential land” (1980-1990), “decreasing population and expanding residential land” (1990-2000), “decreasing population and expanding residential land” (2000-2010), and “decreasing population and decreasing residential land” (2010-2020) from the free distribution stage, free circulation stage of the use right, the paid use stage, to the conditional circulation stage of residential land. The human-land interaction structure in rural residential land varied from intense to gentle. While the socio-economic transformation promoted the expansion of rural residential land. The redevelopment of inefficient use of rural construction land was then promoted the intensive use of rural residential land. 2) There were disparate variations in the human-land interaction of rural residential land in the various districts. Specifically, four districts also presented a significant imbalance structure of “rural population decreasing and land increasing” (i.e., Pudong New Area, Minhang, Jiading, and Baoshan District). There was a strong negative decoupling state between rural settlements and population. These suburban districts shared the more developed informal land market, of which the rural residential land was no longer regarded as a self-occupied place. Their land scale was also expanding rapidly. 3) The dominant factors were determined for the expansion of rural settlements, such as the intensity of land use, urbanization, and regional total population. While the urban-rural population structure was the main hindrance factor. No rigid constraint of land use was attributed to the low level of the intensive use of rural residential land. Urbanization and industrialization also led to represent the housing asset value. The new population in the rural areas was then produced the new housing demand. All factors resulted in the scale expansion of rural residential land. The urban and rural population structure promoted the transfer of rural residential land, thus impeding the further expansion of rural settlements. The driving factors shared significant spatial heterogeneity. An approval system was required for the rural housing construction in the study areas. Strict regulations can be released to restrict the area standards for the free use of homestead land. In addition, the appropriate economic tools (such as taxes and penalties) can also be considered to control the unapproved and unorganized expansion of rural settlements. These findings are of great significance to alleviate the contradiction between human-land interaction, in order to promote the efficient utilization of rural land resources in sustainable agriculture.