Typically, materials with large optical losses such as metals are used as microheaters for silicon based thermo-optic phase shifters. Consequently, the heater must be placed far from the waveguide, which could come at the expense of the phase shifter performance. Reducing the gap between the waveguide and the heater allows reducing the power consumption or increasing the switching speed. In this work, we propose an ultra-low loss microheater for thermo-optic tuning by using a CMOS-compatible transparent conducting oxide such as indium tin oxide (ITO) with the aim of drastically reducing the gap. Using finite element method simulations, ITO and Ti based heaters are compared for different cladding configurations and TE and TM polarizations. Furthermore, the proposed ITO based microheaters have also been fabricated using the optimum gap and cladding configuration. Experimental results show power consumption to achieve a π phase shift of 10 mW and switching time of a few microseconds for a 50 µm long ITO heater. The obtained results demonstrate the potential of using ITO as an ultra-low loss microheater for high performance silicon thermo-optic tuning and open an alternative way for enabling the large-scale integration of phase shifters required in emerging integrated photonic applications.