Gut inflammation involves contributions from immune and non-immune cells, whose interactions are shaped by the spatial organization of the healthy gut and its remodeling during inflammation. The crosstalk between fibroblasts and immune cells is an important axis in this process, but our understanding has been challenged by incomplete cell-type definition and biogeography. To address this challenge, we used MERFISH to profile the expression of 940 genes in 1.35 million cells imaged across the onset and recovery from a mouse colitis model. We identified diverse cell populations; charted their spatial organization; and revealed their polarization or recruitment in inflammation. We found a staged progression of inflammation-associated tissue neighborhoods defined, in part, by multiple inflammation-associated fibroblasts, with unique expression profiles, spatial localization, cell-cell interactions, and healthy fibroblast origins. Similar signatures in ulcerative colitis suggest conserved human processes. Broadly, we provide a framework for understanding inflammation-induced remodeling in the gut and other tissues.