Abstract: Flexible insulated wall solar greenhouse (FIWSG) represents an upgraded improvement over traditional soil-walled solar greenhouse. However, due to the replacement of thick soil walls by steel tubes with large slenderness ratio, structural stability issues caused by insufficient stiffness become more prominent when developing towards larger span. To address these challenges, this study synthesized the mechanical characteristics of existing diagonal and strut brace systems and proposed a Y-type brace suitable for the arches used in FIWSGs. This method aimed to enhance stability capacity and reduce steel consumption by adjusting the local stiffness and internal force transmission paths of FIWSG. Based on this, a static analysis was conducted on the Y-type braced solid-web FIWSG. Subsequently, a variable-stiffness truss arch for FIWSG was proposed, which arranged the upper chord, lower chord, and web members according to the bending moment and deformation distribution diagrams. Using elastoplastic mechanics and nonlinear finite element (FE) method, the general-purpose FE analysis software ANSYS was employed, with the Beam188 element from its element library simulating the greenhouse members. The member base and the foundation were connected with a fixed constraint. The arc-length method, which could trace the descending branch of the equilibrium path, was employed to account for geometric nonlinearity. Additionally, for the material nonlinearity parameter settings, the constitutive relationship adopted a bilinear model, the von Mises yield criterion, and the BKIN bilinear kinematic hardening model. Refined FE analysis models were established for FIWSGs with different brace types (no brace, diagonal brace, strut brace, Y-type brace, 3/4-2.2Y-type brace, 27/40-2.2Y-type brace, 3/5-2.2Y-type brace, 3/5-2.7Y-type brace, and 3/5-3.2Y-type brace) and different arch forms (solid-web and truss). Extensive elastoplastic stability analyses and parametric studies were carried out. By comparatively analyzing the comprehensive performance—including stability capacity and steel consumption indicators—the critical span applicable for arch selection in FIWSGs was obtained for the first time. The analysis results indicate: (1) Compared to other brace types, Y-type brace demonstrates the greatest improvement in the comprehensive performance of the arch. By comparing different bifurcation heights and front roof support point position, the 3/5-2.2Y-type and 27/40-2.2Y-type braces show the maximum increase in stability capacity, with the 27/40-2.2Y-type brace exhibiting superior overall performance. (2) The overall displacement of the 3/5-2.2Y-type brace is reduced by 10.3%, 19.8%, and 30.4% compared to scenarios with no brace, only diagonal brace and only strut brace, respectively, and the instability location gradually shifts toward the front wall. By comparing changes in arch stiffness and internal force transmission processes among diagonal brace, strut brace, and Y-type brace, the internal reasons for the improved stability capacity are explained from the perspective of mechanical mechanism. (3) Through comparative analysis of the stability capacity and steel consumption of FIWSGs with spans of 10 m, 12 m, 14 m, 16 m, 18 m, and 20 m and different arch forms, the critical span between solid-web and truss arches is determined. Specifically, when the span does not exceed 14 m, solid-web arches demonstrate better comprehensive performance, whereas when the span exceeds 14 m, truss arches exhibit superior comprehensive performance. The methods proposed and conclusions drawn in this work provide theoretical references and technical guidance for performance enhancement, arch selection, and engineering practice of FIWSGs.