The gaussian function is simple and easy to implement as Point Spread Function (PSF) model for fitting the position of fluorescent emitters in localization microscopy. Despite its attractiveness the appropriateness of the gaussian is questionable as it is not based on the laws of optics. Here we study the effect of emission dipole orientation in conjunction with optical aberrations on the localization accuracy of position estimators based on a gaussian model PSF. Simulated image spots, calculated with all effects of high numerical aperture, interfaces between media, polarization, dipole orientation and aberrations taken into account, were fitted with a gaussian PSF based Maximum Likelihood Estimator. For freely rotating dipole emitters it is found that the gaussian works fine. The same, theoretically optimum, localization accuracy is found as if the true PSF were a gaussian, even for aberrations within the usual tolerance limit of high-end optical imaging systems such as microscopes (Marechal's diffraction limit). For emitters with a fixed dipole orientation this is not the case. Localization errors are found that reach up to 40 nm for typical system parameters and aberration levels at the diffraction limit. These are systematic errors that are independent of the total photon count in the image. The gaussian function is therefore inappropriate, and more sophisticated PSF models are a practical necessity.