The human brain manages to correctly interpret almost every visual image it receives from the environment. Underlying this ability are contour grouping mechanisms that appropriately link local edge elements into global contours. Although a general view of how the brain achieves effective contour grouping has emerged, there have been a number of different specific proposals and few successes at quantitatively predicting performance. These previous proposals have been developed largely by intuition and computational trial and error. A more principled approach is to begin with an examination of the statistical properties of contours that exist in natural images, because it is these statistics that drove the evolution of the grouping mechanisms. Here we report measurements of both absolute and Bayesian edge co-occurrence statistics in natural images, as well as human performance for detecting natural-shaped contours in complex backgrounds. We find that contour detection performance is quantitatively predicted by a local grouping rule derived directly from the co-occurrence statistics, in combination with a very simple integration rule (a transitivity rule) that links the locally grouped contour elements into longer contours.