Residential buildings take a large share of the globally accumulated construction material stock. Knowledge regarding material stocks and flows and the spatial and temporal dynamics of residential structures is needed to enhance material circularity and, consequently, reduce the inflow of raw materials and the outflow of demolition waste. In this study, material stocks and flows in residential buildings are estimated with the aim of identifying spatiotemporal characteristics among different settlements. Sweden’s 290 municipalities classified into eight settlement types are used as a case study. The modeling is realized bottom-up, with statistical inventory data and material intensity coefficients. The results demonstrate that, in the last 25 years, 63% of the residential material stock in Sweden has accumulated in the three largest metropolitan areas (Stockholm, Gothenburg, and Malmö). In contrast, rural areas and the commuter municipalities to small cities show a decrease in material stocks. For the same period, particularities in material flows were also depicted. For instance, medium-size towns presented the highest inflow and outflow, commuter municipalities to large cities showed no outflow, and rural areas with a visiting industry had a high stock renewal. The material composition differs among settlements due to fluctuations in residential building type and age distribution of the stock. Brick and steel have higher stock in larger settlements, whereas wood and aggregate show higher stock in smaller, commuter and rural settlements. Spatially, it was observed that residential stock mainly accumulates in large cities located near the sea, and inland in medium-size towns along large transportation infrastructures such as high-speed roads that connect the large cities. These spatial observations indicate the formation of an intensively built human corridor, which can be used for enhancing material circularity within the national boundaries. The results point to spatial and temporal particularities in material stocks and flows among settlement types and, therefore, further related research is recommended. This study also contributes with an extensive discussion on drivers of stock accumulation, policy implications, uncertainties, and results validation.