Despite predictions of the classic, hybrid-sterility model of chromosomal speciation, some organisms demonstrate high rate of karyotype evolution. This rate is especially impressive in Agrodiaetus butterflies that rapidly evolved the greatest chromosome number diversity known in animal kingdom within a single subgenus. Here we analyzed karyotype evolution in Agrodiaetus using phylogenetic comparative methods. We found that chromosome numbers possess a strong phylogenetic signal. This disproves the chromosome megaevolution model that proposes multiple chromosome rearrangements to accumulate independently in each of closely related species. We found that Brownian motion gives a more adequate description of observed trait changes than Ornstein-Uhlenbeck model. This indicates that chromosome numbers evolve via random walk along branches of the phylogeny. We discovered a correlation between karyotype changes and phylogeny branch lengths. This gradual pattern is inconsistent with the hybrid-sterility model which, due to association of major chromosome changes with cladogenetic events, predicts a high degree of punctualism in karyotype evolution. Thus, low underdominace of chromosomal rearrangements and/or prevalence of the recombination-suppression model over the hybrid-sterility model of chromosome speciation are the most common engines of the runaway chromosome number change observed.