Intensification or abandonment of agricultural land use has led to a severe decline of semi-natural habitats across Europe. This can cause immediate loss of species but also time-delayed extinctions, known as the extinction debt. In a pan-European study of 147 fragmented grassland remnants, we found differences in the extinction debt of species from different trophic levels. Present-day species richness of long-lived vascular plant specialists was better explained by past than current landscape patterns, indicating an extinction debt. In contrast, short-lived butterfly specialists showed no evidence for an extinction debt at a time scale of c. 40 years. Our results indicate that management strategies maintaining the status quo of fragmented habitats are insufficient, as time-delayed extinctions and associated co-extinctions will lead to further biodiversity loss in the future.