Owing to their crucial roles in regulating protein function, phosphorylation sites (phosphosites) are expected to be evolutionarily conserved. However, mixed results regarding this prediction have been reported. We resolve these contrasting conclusions to show that phosphosites are, on average, more conserved than non-phosphorylated equivalent residues when their enrichment in disordered regions of proteins is taken into account. Phosphosites of known function are dramatically more conserved than those with no characterized function, indicating that the apparent rapid evolution of phosphoproteomes results from a large fraction of phosphosites being non-functional. Our findings highlight the need to use evolutionary information to identify functional regulatory features such as post-translational modifications of eukaryotic proteomes.