Mutation is the source of both beneficial adaptive variation and deleterious genetic load, fueling the opposing selective forces than shape mutation rate evolution. This dichotomy is well illustrated by the evolution of the mutator phenotype, a genome-wide 10- to 100-fold increase in mutation rate. This phenotype has often been observed in clonally expanding populations exposed to novel or frequently changing conditions. Although studies of both experimental and natural populations have shed light on the evolutionary forces that lead to the spread of the mutator allele through a population, significant gaps in our understanding of mutator evolution remain. Here we use an experimental evolution approach to investigate the conditions required for the evolution of a reduction in mutation rate and the mechanisms by which populations tolerate the accumulation of deleterious mutations. We find that after ∼6,700 generations, four out of eight experimental mutator lines had evolved a decreased mutation rate. We provide evidence that the accumulation of deleterious mutations leads to selection for reduced mutation rate clones in populations of mutators. Finally, we test the long-term consequences of the mutator phenotype, finding that mutator lines follow different evolutionary trajectories, some of which lead to drug resistance.