Inferring the history of speciation in house mice from autosomal, X-linked, Y-linked and mitochondrial genes

Patterns of genetic differentiation among taxa at early stages of divergence provide an opportunity to make inferences about the history of speciation. Here, we conduct a survey of DNA-sequence polymorphism and divergence at loci on the autosomes, X chromosome, Y chromosome and mitochondrial DNA in samples of Mus domesticus, M. musculus and M. castaneus. We analyzed our data under a divergence with gene flow model and estimate that the effective population size of M. castaneus is 200 000–400 000, of M. domesticus is 100 000–200 000 and of M. musculus is 60 000–120 000. These data also suggest that these species started to diverge approximately 500 000 years ago. Consistent with this recent divergence, we observed considerable variation in the genealogical patterns among loci. For some loci, all alleles within each species formed a monophyletic group, while at other loci, species were intermingled on the phylogeny of alleles. This intermingling probably reflects both incomplete lineage sorting and gene flow after divergence. Likelihood ratio tests rejected a strict allopatric model with no gene flow in comparisons between each pair of species. Gene flow was asymmetric: no gene flow was detected into M. domesticus, while significant gene flow was detected into both M. castaneus and M. musculus. Finally, most of the gene flow occurred at autosomal loci, resulting in a significantly higher ratio of fixed differences to polymorphisms at the X and Y chromosomes relative to autosomes in some comparisons, or just the X chromosome in others, emphasizing the important role of the sex chromosomes in general and the X chromosome in particular in speciation.