The chemically reduced graphene oxide (rGO) was prepared by the reduction of graphene oxide by hydrazine hydrate. By varying the reduction time (10 min, 1 h, and 15 h), oxygen functional groups on rGO were tremendously controlled and they were named RG1, RG2, and RG3, respectively. Here, we investigate the impact of oxygen functional groups on the detection of ammonia and toluene at room temperature. Their effect on sensing mechanism was analyzed by first-principles calculation-based density functional theory. The sensing material was fabricated, and the effect of reduction time shown improved the recovery of ammonia and toluene sensing at room temperature. Structural, morphological, and electrical characterizations were performed on both RG1 and RG3. The sensor response toward toluene vapor of 300 ppm was found to vary 4.4, 2.5, and 3.8% for RG1, RG2, and RG3, respectively. Though RG1 shows higher sensing response with poor recovery, RG3 exhibited complete desorption of toluene after the sensing process with response and recovery times of approximately 40 and 75 s, respectively. The complete recovery of toluene molecules on RG3 is due to the generation of new sites after the reduction of oxygen functionalities on its surface. It could be suggested that these sites provided anchor to ammonia and toluene molecules and good recovery under N2 purge. Both theoretical and experimental studies revealed that tuning the oxygen functional groups on rGO could play a vital role in the detection of volatile organic compounds (VOCs) on rGO sheets and was discussed in detail. This study could provoke knowledge about rGO-based sensor dependency with oxygen functional groups and shed light on effective monitoring of VOCs under ambient conditions for air quality monitoring applications.