In many prokaryotes and eukaryotes, DNA methylation at cis-regulatory sequences determines whether gene expression is on or off. Stable inheritance of these expression states is required in bacterial pathogenesis, cancer and developmental pathways. Here we delineate the factors that control the stability of these states by using the agn43 gene in Escherichia coli as a model system. Systematic disruption of this system shows that a functional switch requires the presence of several, rarely occupied, intermediate states that separate the 'on' and 'off' states. Cells that leave the on and off state enter different intermediate states, where there is a strong bias that drives cells back to their original state. The intermediate states therefore act as buffers that prevent back and forth switching. This mechanism of generating multiple states is an alternative to feedback regulation, and its general principle should be applicable to the analysis of other epigenetic switches and the design of synthetic circuits.