An influential causal theory attributes dyslexia to visual and/or auditory perceptual deficits. This theory derives from group differences between individuals with dyslexia and controls on a range of psychophysical tasks, but there is substantial variation, both between individuals within a group and from task to task. We addressed two questions. First, do psychophysical measures have sufficient reliability to assess perceptual deficits in individuals? Second, do different psychophysical tasks measure a common underlying construct? We studied 104 adults with a wide range of reading ability and two comparison groups of 49 dyslexic adults and 41 adults with normal reading, measuring performance on four auditory and two visual tasks. We observed moderate to high test-retest reliability for most tasks. While people with dyslexia were more likely to display poor task performance, we were unable to demonstrate either construct validity for any of the current theories of perceptual deficits or predictive validity for reading ability. We suggest that deficient perceptual task performance in dyslexia may be an associated (and inconsistent) marker of underlying neurological abnormality, rather than being causally implicated in reading difficulties.