Unbiased search on the potential energy surface of medium-sized boron clusters, with B(80), B(74), and B(68) as representatives, was carried out using simulated annealing incorporated with first-principles molecular dynamics. These boron clusters thermodynamically prefer the B(12)-centered core-shell structures, which resemble the fragment of bulk boron solids. Though these core-shell clusters lack a descriptive symmetry and may not be the true global minima, the core-shell B(80) is about 25 meV/atom lower in energy than the widely assumed highly stable "magic" B(80) fullerene. The electronic states and photoelectron spectra of these clusters are closely correlated to the structural motif, which may be helpful for detecting the cluster configurations in experiments.