The development of a microwave electrometer with inherent uncertainty approaching its ultimate limit carries both fundamental and technological significance. However, because of the thermal motion of atoms, the state-of-art Rydberg electrometer falls considerably short of the standard quantum limit by about three orders of magnitude. Here, we use an optically thin medium with approximately 5.2 × 105 laser-cooled atoms to implement the microwave heterodyne detection. By mitigating various noises and strategically optimizing the electrometer parameters, our study reduces the equivalent noise temperature by a factor of 20 and achieves an electric field sensitivity of 10.0 nV cm-1 Hz-1/2, lastly reaching a factor of 2.6 above the standard quantum limit. Our work also provides valuable insights into the inherent capabilities and limitations of Rydberg electrometers, offering superior sensitivity in detecting weak microwave signals for numerous applications.