Certain types of cellular differentiation are probabilistic and transient. In such systems individual cells can switch to an alternative state and, after some time, switch back again. In Bacillus subtilis, competence is an example of such a transiently differentiated state associated with the capability for DNA uptake from the environment. Individual genes and proteins underlying differentiation into the competent state have been identified, but it has been unclear how these genes interact dynamically in individual cells to control both spontaneous entry into competence and return to vegetative growth. Here we show that this behaviour can be understood in terms of excitability in the underlying genetic circuit. Using quantitative fluorescence time-lapse microscopy, we directly observed the activities of multiple circuit components simultaneously in individual cells, and analysed the resulting data in terms of a mathematical model. We find that an excitable core module containing positive and negative feedback loops can explain both entry into, and exit from, the competent state. We further tested this model by analysing initiation in sister cells, and by re-engineering the gene circuit to specifically block exit. Excitable dynamics driven by noise naturally generate stochastic and transient responses, thereby providing an ideal mechanism for competence regulation.