We present a comparative study on an experimental and theoretical optimization of magneto-optical Kerr setups based on photoelastic modulation and phase sensitive detector methodology. The first and second harmonics, I omega,2 omega, of the reflected light intensity are measured for a CoO/Co magnetic reference film. The magnetic field dependence of the optical off-diagonal Fresnel reflection coefficients rps and rsp follows the sample magnetization. Different Kerr setups provide various dependencies of I omega,2 omega on the reflection coefficients and, hence, on the Kerr ellipticity epsilon K and rotation theta K. Jones matrix formalism has been used to analyze the impact of a systematic variation of relative analyzer and polarizer orientations with respect to each other and with respect to the retardation axis of the modulator involved in longitudinal Kerr setups for incoming s-polarized light. We find one particular setup which maximizes I(omega) as well as I2 omega and maximizes the signal-to-noise ratio. Inefficient setups are characterized by I omega,2 omega intensities involving large nonmagnetic contributions of rp and rs.