Brain function is reflected in connectome community structure. The dominant view is that communities are assortative and segregated from one another, supporting specialized information processing. However, this view precludes the possibility of non-assortative communities whose complex inter-community interactions could engender a richer functional repertoire. We use weighted stochastic blockmodels to uncover the meso-scale architecture of Drosophila, mouse, rat, macaque, and human connectomes. We find that most communities are assortative, though others form core-periphery and disassortative structures, which better recapitulate observed patterns of functional connectivity and gene co-expression in human and mouse connectomes compared to standard community detection techniques. We define measures for quantifying the diversity of communities in which brain regions participate, showing that this measure is peaked in control and subcortical systems in humans, and that inter-individual differences are correlated with cognitive performance. Our report paints a more diverse portrait of connectome communities and demonstrates their cognitive relevance.