For the memory application of magnetoelectric multiferroics, not only bistability (i.e., ferroelectricity) but also the switching of the polarization direction with some noneverlasting stimulus is necessary. Here, we report a novel method for the electric polarization reversal in TbMnO3 without the application of an electric field or heat. The direction of the magnetic-field-induced polarization along the a axis (Pa) is memorized even in the zero field where Pa is absent. The polarization direction can be reversed by rotating the magnetic-field direction in the ab plane.