A multivariate classification approach has been presented to examine the brain abnormalities, i.e., due to prenatal cocaine exposure, using both structural and functional brain images. First, a regional statistical feature extraction scheme was adopted to capture discriminative features from voxel-wise morphometric and functional representations of brain images, in order to reduce the dimensionality of the features used for classification, as well as to achieve the robustness to registration error and inter-subject variations. Then, this feature extraction method was used in conjunction with a hybrid feature selection method and a nonlinear support vector machine for the classification of brain abnormalities. This brain classification approach has been applied to detecting the brain abnormality associated with prenatal cocaine exposure in adolescents. A promising classification performance was achieved on a data set of 49 subjects (24 normal and 25 prenatally cocaine-exposed teenagers), with a leave-one-out cross-validation. Experimental results demonstrated the efficacy of our method, as well as the importance of incorporating both structural and functional images for brain classification. Moreover, spatial patterns of group difference derived from the constructed classifier were mostly consistent with the results of the conventional statistical analysis method. Therefore, the proposed approach provided not only a multivariate classification method for detecting brain abnormalities, but also an alternative way for group analysis of multimodality images.