Bacterial opportunistic pathogens are feared for their difficult-to-treat nosocomial infections and for causing morbidity in immunocompromised patients. Here, we study how such a versatile opportunist, Pseudomonas aeruginosa, adapts to conditions inside and outside its model host Caenorhabditis elegans, and use phenotypic and genotypic screens to identify the mechanistic basis of virulence evolution. We found that virulence significantly dropped in unstructured environments both in the presence and absence of the host, but remained unchanged in spatially structured environments. Reduction of virulence was either driven by a substantial decline in the production of siderophores (in treatments without hosts) or toxins and proteases (in treatments with hosts). Whole-genome sequencing of evolved clones revealed positive selection and parallel evolution across replicates, and showed an accumulation of mutations in regulator genes controlling virulence factor expression. Our study identifies the spatial structure of the non-host environment as a key driver of virulence evolution in an opportunistic pathogen.