In this paper, we present an automatic method for estimating the trajectories of Escherichia coli bacteria from in vivo phase-contrast microscopy. To address the low-contrast boundaries in cellular images, an adaptive kernel-based technique is applied to detect cells in each frame. In addition to intensity features, region homogeneity measure and class uncertainty are also applied in this detection technique. To track cells with complex motion, a novel matching gain measure is introduced to cope with the challenges, particularly the presence of low-contrast boundary, the variations of appearance, and the frequent overlapping and occlusion. For multicell tracking over time, an optimal matching strategy is introduced to improve the handling of cell collision and broken trajectories. The results of successful tracking of Escherichia coli from various phase-contrast sequences are reported and compared with manually determined trajectories, as well as those obtained from existing tracking schemes. The stability of the algorithm with different parameter values is also analyzed and discussed.