We study and realize asymmetric fiber-based cavities with optimized mode match to achieve high reflectivity on resonance. This is especially important for mutually coupling two physical systems via light fields, e.g., in quantum hybrid systems. Our detailed theoretical and experimental analysis reveals that on resonance, the interference effect between the directly reflected non-modematched light and the light leaking back out of the cavity can lead to large unexpected losses due to the mode filtering of the incoupling fiber. Strong restrictions for the cavity design result out of this effect and we show that planar-concave cavities are clearly best suited. We validate our analytical model using numerical calculations and demonstrate an experimental realization of an asymmetric fiber Fabry-Pérot cavity with optimized parameters.