We propose a multi-target tracking method using an Interacting Multiple Model Joint Probabilistic Data Association (IMM-JPDA) filter for tracking vesicles in total internal reflection fluorescence microscopy (TIRFM) sequences. We enhance the accuracy and reliability of the algorithm by tailoring an appropriate framework to this application. Evaluation of our algorithm is performed on both realistic synthetic data and real TIRFM data. Our results are compared against related methods and a commercial tracking software.