RT Journal Article
SR Electronic
T1 Image memorability is predicted at different stages of a convolutional neural network
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 834796
DO 10.1101/834796
A1 Griffin E. Koch
A1 Essang Akpan
A1 Marc N. Coutanche
YR 2020
UL http://biorxiv.org/content/early/2020/03/14/834796.abstract
AB The features of an image can be represented at multiple levels – from low-level visual properties to high-level meaning. What factors drive some images to be memorable while others are forgettable? Across three behavioral experiments, we addressed this question. In a first behavioral experiment, we combined a convolutional neural network (CNN) with behavioral prospective assignment, by using four CNN layers to select the scene images that each of one hundred participants experience. We found that participants remembered more images when they were assigned to view stimuli selected based on their discriminability in low-level CNN layers, or similarity in high-level layers. A second experiment replicated these results in an independent sample of participants. The third experiment investigated how memorability is influenced when images fall within a single semantic category (houses). We replicated results from the first two experiments at lower levels, but found that similarity predicted memorability at a mid-high level, rather than the highest level observed for scenes from multiple categories. This mid-high level contains representations for objects and object-parts, which are more important for discriminating images from the same category. Together, this research provides evidence that discriminability at different visual levels, modeled using a CNN, predicts image memorability.