The study aims to examine whether the variation of fasting plasma glucose (FPG), represented by coefficient of variation (CV), independently predicts all-cause mortality among Chinese type 2 diabetes patients. This retrospective cohort study was designed based on a standardized electronic management system of diabetes patients in Shanghai, China. 8871 type 2 diabetes patients were enrolled between 1 January 2007 and 31 December 2007 and were followed-up for all-cause mortality until 31 December 2014. All patients were grouped by the quartiles of CV of FPG. 1136 patients deceased during following-up. After adjusting for other risk factors, CV of FPG was not independently associated with all-cause mortality. Stratified analysis by mean FPG levels (<7 mmol/L and ≥7 mmol/L) observed a significant modifying effect of CV of FPG (P for interact test <0.01). CV of FPG was independently associated with all-cause mortality in patients whose glucose control was poor, with the HRs (95% CI) for the second, third, fourth vs first quartiles of CV of FPG being 1.23(0.94-1.61), 1.23(0.94-1.61), and 1.63(1.25-2.13), respectively. Our results suggest that variability of FPG may be an important predictor of mortality among type 2 diabetes in China, particularly for those with their glycemic status uncontrolled.