We apply time-resolved MeV electron diffraction to study the electron-lattice energy relaxation in thin film Au-insulator heterostructures. Through precise measurements of the transient Debye-Waller-factor, the mean-square atomic displacement is directly determined, which allows to quantitatively follow the temporal evolution of the lattice temperature after short pulse laser excitation. Data obtained over an extended range of laser fluences reveal an increased relaxation rate when the film thickness is reduced or the Au-film is capped with an additional insulator top-layer. This behavior is attributed to a cross-interfacial coupling of excited electrons in the Au film to phonons in the adjacent insulator layer(s). Analysis of the data using the two-temperature-model taking explicitly into account the additional energy loss at the interface(s) allows to deduce the relative strength of the two relaxation channels.