Voltage-activated ion channels open and close in response to changes in voltage, a property that is essential for generating nerve impulses. Studies on voltage-activated potassium (Kv) channels show that voltage-sensor activation is sensitive to the composition of lipids in the surrounding membrane. Here we explore the interaction of lipids with S1-S4 voltage-sensing domains and find that the conversion of the membrane lipid sphingomyelin to ceramide-1-phosphate alters voltage-sensor activation in an S1-S4 voltage-sensing protein lacking an associated pore domain, and that the S3b-S4 paddle motif determines the effects of lipid modification on Kv channels. Using tarantula toxins that bind to paddle motifs within the membrane, we identify mutations in the paddle motif that weaken toxin binding by disrupting lipid-paddle interactions. Our results suggest that lipids bind to voltage-sensing domains and demonstrate that the pharmacological sensitivities of voltage-activated ion channels are influenced by the surrounding lipid membrane.