The miniature ultracapacitors, with interdigitated electrodes of vertically aligned carbon nanotubes (VACNTs) and an inter-electrode gap of 20 μm, have been prepared in the LiPF6 organic electrolyte with and without PVdF-HFP gel. PVdF-HFP between two opposing electrodes enhances the device reliability, but lessens its power performance because of the extra diffusion resistance. Also noteworthy are the gel influences on the cycle stability. When the applied voltage is 2.0 or 2.5 V, both the LiPF6 and the gel capacitors exhibit excellent stability, typified by a retention ratio of ≥95% after 10,000 cycles. Their coulombic efficiencies quickly rise up, and hold steady at 100%. Nonetheless, when the applied voltage is 3.5 or 4.0 V, the cycle stability deteriorates, since the negative electrode potential descends below 0.9 V (vs. Li), leading to electrolyte decomposition and SEI formation. For the LiPF6 capacitor, its retention ratio could be around 60% after 10,000 cycles and the coulombic efficiency of 100% is difficult to reach throughout its cycle life. On the other hand, the gel capacitor cycles energy with a much higher retention ratio, >80% after 10,000 cycles, and a better coulombic efficiency, even though electrolyte decomposition still occurs. We attribute the superior stability of the gel capacitor to its extra diffusion resistance which slows down the performance deterioration.