Segmenting cortical and sub-cortical structures from 3D brain images is of significant practical importance. However, various anatomical structures have similar intensity patterns in MRI, and the automatic segmentation of them is a challenging task. In this paper, we present a new brain segmentation algorithm using a hybrid model. (1) A multi-class classifier, PBT.M2, is proposed for learning/computing multi-class discriminative models. The PBT.M2 handles multi-class patterns more easily than the original probabilistic boosting tree (PBT), and it facilitates the process, eventually, toward whole brain segmentation. (2) We use an edge field, by learning, to constraint the region boundaries. We show the improvements due to the two new aspects both numerically and visually, and also compare the results with those by FreeSurfer. Our algorithm is general and easy to use, and the results obtained are encouraging.