Smelling sensations: olfactory crossmodal correspondences

Crossmodal correspondences are the associations between apparently distinct stimuli in different sensory modalities. These associations, albeit surprising, are generally shared in most of the population. Olfaction is ingrained in the fabric of our daily life and constitutes an integral part of our perceptual reality, with olfaction being more commonly used in the entertainment and analytical domains, it is crucial to uncover the robust correspondences underlying common aromatic compounds. Towards this end, we investigated an aggregate of crossmodal correspondences between ten olfactory stimuli and other modalities (angularity of shapes, smoothness of texture, pleasantness, pitch, colours, musical genres and emotional dimensions) using a large sample of 68 observers. We uncover the correspondences between these modalities and extent of these associations with respect to the explicit knowledge of the respective aromatic compound. The results revealed the robustness of prior studies, as well as, contributions towards olfactory integration between an aggregate of other dimensions. The knowledge of an odour’s identity coupled with the multisensory perception of the odours indicates that these associations, for the most part, are relatively robust and do not rely on explicit knowledge of the odour. Through principal component analysis of the perceptual ratings, new cross-model mediations have been uncovered between odours and their intercorrelated sensory dimensions. Our results demonstrate a collective of associations between olfaction and other dimensions, potential cross modal mediations via exploratory factor analysis and the robustness of these correspondence with respect to the explicit knowledge of an odour. We anticipate the findings reported in this paper could be used as a psychophysical framework aiding in a collective of applications ranging from olfaction enhanced multimedia to marketing.


Introduction 31
Olfaction is ingrained into the fabric of our lives, altering the very perception of our favourite 32 commodities and plays a crucial role in the multi-sensory perception of our surrounding environment.

33
Exploring the crossmodal correspondences provided by the chemical senses is pivotal towards solving 34 the crossmodal binding problem, and in turn, their implication on interactive and immersive 35 experiences. Crossmodal correspondences can be depicted as the consistent correspondence between 36 stimulus features in different sensory modalities [1]. These correspondences can be matched if they 37 both have the same effect on the observers' mood, emotional state, alertness and/or arousal [1][2][3].

38
Aromas are consistently perceived together with other stimuli principally visual, the absence of these 39 additional stimuli results in inferior identification [4,5]. Albeit, semantic congruency can enhance the 40 perceived pleasantness [6,7], discrimination [8] and correct identification of odours [9]. Crossmodal 41 correspondences have been shown to induce a bias (i.e., providing a red glass of white wine can bias 42 the judgment of expert wine tasters [10]). In terms of evolution, olfaction is one of the oldest senses and 43 plays a major role in social behaviour, communication and emotional evaluation; mood and emotional 44 processes share a common neural substrate with the olfactory pathway, namely the limbic system [11].

45
It is therefore likely that olfactory information plays a major role in modulating the quality of our 46 immersive multisensorial experiences.

47
Crossmodal interactions between smell and both vision and hearing have been of distinct 48 interest, as it has been shown to alter olfactory perception considerably. For example, olfaction-audition contributory factor to idiosyncratic, as opposed to robust associations, is the relatively high inter-55 observer variability of the olfactory sense [29]. There is an increasing demand to provide the chemical time to ensure approximately uniform evaporation. The odours were replaced every two weeks.

111
A nine-point scale was constructed with a rounded shape "bouba" and an angular shape "kiki" 112 on the left and right side of the scale respectively. Similar to an earlier experiment performed by [20].

113
The midpoint of the nine-point scale (5)

124
The full range of audible frequencies (20Hz to 20kHz) was implemented using a slider where 125 movement from left to right corresponded to an increase in frequency. Every time the slider was 126 adjusted the respective frequency was played, producing a sinusoidal tone lasting 1 second in length.

127
Due to the large volume of potential selections, participants were played a sample from each end of the 128 scale, followed by a sample at 10kHz, if the current pitch didn't match the odour a lower or higher pitch 129 was selected (approximately half way between the last two frequencies played) as indicated by the 130 participant.

Music Stimuli
132 Seven different music genres; classical, country, heavy metal, jazz, rap, classic rock, and soul.

133
Five were selected from [34] with an additional two added due to their wide popularity. Each sample 134 was 15 seconds in duration and played at the same volume across participants. Participants had to listen 135 to each sample at least once during the questions first occurrence, the order was subject to the 136 participants preference.

138
The CIE L*a*b* colour space was used because of its perceptual uniformity, participants could 139 slide through 101 linear interpolated slices from the L* channel of the colour space increasing or 140 decreasing the lightness. Only colours that fit in sRGB colour gamut were shown. This removed the 141 limitations of earlier studies that let participants choose from a small selection of colours.

143
A subset of emotions from the Universal Emotion and Odour Scale [9] was included these 144 where; angry, aroused, bored, calm, disgust, excited, happy, sad and scared, additionally an option for 145 neutral (no opinion) was added to negate a tentative assignment.

147
A list of different aromas was compiled consisting of the ten odours used in this experiment 148 along with an additional fifteen and presented in alphabetical order. This was incorporated to increase 149 the number of choices the participant had available, so they were less likely to make inferred decisions 150 when trying to identify the current odour. The identification task was at the end of the experiment. that the odours significantly affected all ratings; angularity (x 2 (9) = 122.15, P < 0.05), smoothness (x 2 (9)

159
Bonferroni multiple comparison test was conducted to identify which odours where significantly 160 different from each other. A post-hoc one sample t-test with a Bonferroni correct alpha was conducted 161 to determine which of the odours was significantly different from 0 (the original scale's grand mean).

163
The significantly 'angular' odours are peppermint (P < 0.005, t = 8.62) and lemon (P < 0.005, t = 3.43),  and participants emotional response (x 2 = 187.54, P < 0.05). Consequently, chi-squared tests for 183 goodness of fit were conducted to see which of the presented stimuli were significantly different from 184 a chance selection. The odours that were significantly different from a chance selection in the genre 185 association task are black pepper, caramel, cherry, coffee, freshly cut grass, lemon and orange (P < 186 0.05) (See Fig 2A). The odours were significantly different from chance selection in the emotion 187 association task (P < 0.05) (See Fig 2B).

Fig 4. Angularity, smoothness, pleasantness and pitch identification dependencies. Mean z-scores
229 asterisks detonate odours where there is significant variation between the correct and incorrect ratings.

230
The green markers detonate correct classification and the red markers detonate incorrect classification.

231
The error bars show a 95% confidence interval.

233
To access if explicit knowledge of the odour affected the emotional and genre dimensions the 234 relative difference between correct and incorrect identification was calculated for each odour. Fig 5A   235 shows that the knowledge of an odour does affect the emotional dimensions and has Frobenius norm of 236 323.06. Peppermint, for example, was perceived as less happy and angrier. Generally, the odours were 237 perceived as being more neutral, calming and slightly disgusting. From Fig 5B we can see that 238 knowledge of the odours affects the genre dimensions, for example, peppermint is perceived to be less 239 jazz and more metal and rap the genre association matrix has a Frobenius norm of 339.70.

264
(20.59%) (see Fig 7). Retrospective category identification was determined by the participants' ability 265 to pick another odour in the same category following the fragrance classes outlined in [35]. An accuracy rating of 62.94% was achieved for category identification, each potential classification belonged to only 267 one category. A Pearson correlation indicated that there was no strong correlation between the age of 268 the participant and their identification accuracy (P = -0.17).

299
The PCA analysis score plot shows the perceptual similarity between the olfactory stimuli, for 300 example, (lemon, cherry and orange), (lavender, freshly cut grass and pine), (coffee and caramel) 301 obtained similar results in most, but potentially not all, ratings analysed using PCA. The PCA loadings 302 plot suggests that the hedonic values are a strongly influenceable factor, that is, the strong loadings of 303 the pleasant (i.e., happy, excited and calm), unpleasant (i.e., sad, angry and scared) and the 'pleasantness 304 dimension' are at least moderately associated to the other dimensions reported in this paper.

305
Additionally, the loadings plot shows strong associations between the angularity of shapes, textures, 306 pitch, emotional and genre dimensions. Odours judged to be the rounded shape tend to be associated 307 with a smooth texture, be lower in pitch, be more soul. Whereas, the angular shape appears to be 308 strongly associated with the genre dimension rock. Moderate associations exist between the angularity 309 of shapes with the angular shape being associated with lighter colours, being more rap, angry, arousing associations between the odours and identification rate are being more happy, exciting and being less 320 sad and disgusting. Moderate relationships exist between a higher identification rate and being more 321 jazz and less metal, classical, neutral, disgusting and boring. All relationships uncovered in the PCA 322 analysis can also be applied in reverse (i.e., the odours perceived as being smoother are generally 323 assigned to being less angular and more rounded in shape). Consistent with the findings from [37], The

324
PCA shows strong relationships towards the emotional and musical dimensions, for example, the 325 relationships between the emotion 'happy' are being less more jazz and soul and less metal. The 326 relationships between 'angry' are being more metal and less soul.

327
Future work stemming from these findings could include assessing if congruent vs incongruent 328 associations defer any advantages in an objective task. The moderate relations discovered in the PCA 329 analysis, between odours and the different relationships could be explored further with a more tailored