Neuronal computations underlying higher brain functions depend on synaptic interactions among specific neurons. A mechanistic understanding of such computations requires wiring diagrams of neuronal networks. In this study, we examined how the olfactory bulb (OB) performs 'whitening', a fundamental computation that decorrelates activity patterns and supports their classification by memory networks. We measured odor-evoked activity in the OB of a zebrafish larva and subsequently reconstructed the complete wiring diagram by volumetric electron microscopy. The resulting functional connectome revealed an over-representation of multisynaptic connectivity motifs that mediate reciprocal inhibition between neurons with similar tuning. This connectivity suppressed redundant responses and was necessary and sufficient to reproduce whitening in simulations. Whitening of odor representations is therefore mediated by higher-order structure in the wiring diagram that is adapted to natural input patterns.