Polyphenols are an important functional minor component of virgin olive oils that are responsible for the key sensory characteristics of bitterness, pungency, and astringency. Polyphenols were isolated from virgin olive oils by using liquid/liquid extraction and then separated by using reverse phase HPLC followed by fraction collection. The sensory qualities of the isolated polyphenols were evaluated, and almost all fractions containing polyphenols were described as bitter and astringent. However, the fraction containing deacetoxy-ligstroside aglycon produced a strong burning pungent sensation at the back of the throat. In contrast, the fraction containing the analogous deacetoxy-oleuropein aglycon, at an equivalent concentration, produced only a slight burning/numbing sensation, which was perceived more on the tongue. No other polyphenol fractions from the analyzed oils produced the intense burning sensation; thus, deacetoxy-ligstroside aglycon is the polyphenol responsible for the majority of the burning pungent sensation found in pungent extra virgin olive oils.