We investigate gold-4,4'-bipyridine-gold single-molecule junctions with the mechanically controllable break junction technique at cryogenic temperature (T = 4.2 K). We observe bistable probabilistic conductance switching between the two molecular binding configurations, influenced both by the mechanical actuation and by the applied voltage. We demonstrate that the relative dominance of the two conductance states is tunable by the electrode displacement, whereas the voltage manipulation induces an exponential speedup of both switching times. The detailed investigation of the voltage-tunable switching rates provides an insight into the possible switching mechanisms.