Cortical synaptic strengths vary substantially from synapse to synapse and exhibit a skewed distribution with a small fraction of synapses generating extremely large depolarizations. Using multiple whole-cell recordings from rat hippocampal CA3 pyramidal cells, we found that the amplitude of unitary excitatory postsynaptic conductances approximates a lognormal distribution and that in the presence of synaptic background noise, the strongest fraction of synapses could trigger action potentials in postsynaptic neurons even with single presynaptic action potentials, a phenomenon termed interpyramid spike transmission (IpST). The IpST probability reached 80%, depending on the network state. To examine how IpST impacts network dynamics, we simulated a recurrent neural network embedded with a few potent synapses. This network, unlike many classical neural networks, exhibited distinctive behaviors resembling cortical network activity in vivo. These behaviors included the following: 1) infrequent ongoing activity, 2) firing rates of individual neurons approximating a lognormal distribution, 3) asynchronous spikes among neurons, 4) net balance between excitation and inhibition, 5) network activity patterns that was robust against external perturbation, 6) responsiveness even to a single spike of a single excitatory neuron, and 7) precise firing sequences. Thus, IpST captures a surprising number of recent experimental findings in vivo. We propose that an unequally biased distribution with a few select strong synapses helps stabilize sparse neuronal activity, thereby reducing the total spiking cost, enhancing the circuit responsiveness, and ensuring reliable information transfer.