Morphological asymmetries favouring the left hemisphere in the planum temporale (PT) and Heschl's gyrus (HG) have both been presumed to relate to the typical left-hemisphere dominance for language functions. However, a direct link between structure and function has not been clearly established. The present study investigates this issue by measuring the volume of the PT and HG on the MRI scans of epilepsy patients classified into three groups: left speech group (LSG; n = 20), right speech group (RSG; n = 11) and bilateral speech group (BSG; n = 13), as assessed by the intracarotid Sodium Amytal procedure. Additionally, an automatic voxel-based morphometry (VBM) analysis was performed to explore collateral structural asymmetries. Although leftward structural asymmetries were found in the PT, consistent with the literature, they did not relate to language lateralization. For HG we also replicated asymmetries favouring the left side; interestingly, three of the individuals within the RSG showed a strongly reversed asymmetry, but as a whole the structure-function relationship for HG was not obligatory. The VBM analysis revealed a grey-matter concentration difference in the posterior part of the inferior frontal gyrus (pars opercularis, corresponding functionally to Broca's area), which favoured the left hemisphere in the LSG, and the right hemisphere in the RSG. The findings suggest that this frontal cortical region bears a direct relationship to language lateralization, which may be related to use-dependent plasticity in patients with language reorganization.