Anatomical and physiological experiments have outlined a blueprint for the feedforward flow of activity in cortical circuits: signals are thought to propagate primarily from the middle cortical layer (layer 4, L4) up to L2/3 and down to the major cortical output layer (L5). Pharmacological manipulations, however, have contested this model and have suggested that L4 may not be critical for sensory responses of neurons in either superficial or deep layers. To address these conflicting models, we reversibly manipulated L4 activity in awake, behaving mice using cell type-specific optogenetics. In contrast with both prevailing models, we found that activity in L4 directly suppressed L5, in part by activating deep, fast-spiking inhibitory neurons. Our data suggest that the net effect of L4 activity is to sharpen the spatial representations of L5 neurons. Thus, we establish a previously unknown translaminar inhibitory circuit in the sensory cortex that acts to enhance the feature selectivity of cortical output.