Multisensory learning binds modality-specific neurons into a cross-modal memory

14 Associating multiple sensory cues with objects and experience is a fundamental brain process 15 that improves object recognition and memory performance. However, neural mechanisms that 16 bind sensory features during learning and augment memory expression are unknown. Here 17 we demonstrate multisensory appetitive and aversive memory in Drosophila . Combining 18 colors and odors improved memory performance, even when each sensory modality was 19 tested alone. Temporal control of neuronal function revealed visually-selective mushroom 20 body Kenyon Cells (KCs) to be required for both enhancement of visual and olfactory memory 21 after multisensory training. Voltage imaging in head-fixed flies showed that multisensory 22 learning binds activity between streams of modality-specific KCs, so that unimodal sensory 23 input generates a multimodal neuronal response. Binding occurs between regions of the 24 olfactory and visual KC axons, which receive valence-relevant dopaminergic reinforcement, 25 and is propagated downstream. Dopamine locally releases GABA-ergic inhibition to permit 26 specific microcircuits within KC-spanning serotonergic neurons to function as an excitatory 27 bridge between the previously ‘modality-selective’ KC streams. Cross-modal binding thereby 28 expands the olfactory memory engram by recruiting visual path KCs to become odor 29 responsive. This broadening of the engram improves memory performance after multisensory 30 learning and permits a single sensory feature to retrieve the memory of the multimodal

protocol revealed significantly decreased performance compared to that following odor only training (Fig. 1d), suggesting that flies are conflicted when faced with a switch of the odorcolor contingency between training and testing.
To further investigate the nature of multimodal memory enhancement, we restricted the presentation of the multisensory cues to either training or testing.Multisensory training improved memory retrieval even when each sensory modality was presented alone during testing [olfactory retrieval, OR; visual retrieval, VR] (Fig. 1e,f).In contrast, presenting multisensory stimuli only during the memory test, but not during training, did not facilitate performance [multisensory retrieval, MSR] (Extended Data Fig. 1d).Moreover, the greatest improvement in memory performance was observed when multisensory stimuli were used both during training and testing (Extended Data Fig. 1e).Therefore, multisensory training enhances memory performance for the individual odor and color memory components, and congruence of odor and color information between training and testing further enhances performance.Although our experiments and those of others 13,14 imply that flies can distinguish green and blue colors, we do not at this point discount a contribution of hue and luminance.
The dendrites of the numerically larger populations of olfactory KCs within the αβ, α¢β¢ and γ lobes occupy the main calyces of the MB, whereas the relatively small populations of αβ posterior (αβp) and γ dorsal (γd) KCs receive predominantly visual information via their dendrites in the accessory calyces 9 .The γd KCs have previously been implicated in color learning 13,15 .We tested the role of visual γd and αβp KCs in multisensory learning and memory, using cell-specific expression of a UAS-Shibire ts1 (Shi ts1 ) transgene which encodes a dominant temperature-sensitive dynamin 16 .At temperatures >30 ˚C, Shi ts1 blocks membrane recycling and thus impairs synaptic transmission while function can be restored by returning flies to <29˚C.Blocking output from γd and αβp KCs during testing at 6h abolished the visual enhancement of performance in the congruent protocol, and removed the interference effect in the incongruent protocol in both instances memory performance was similar to that of flies tested with odors alone (Fig. 2a-d, Extended Data Fig. 2a-f).These results are consistent with activity in γd and αβp KCs representing the visual component of multisensory memory (see also, Extended Data Fig. 2g-h).Surprisingly, blocking transmission from γd KCs (but not αβp KCs) during testing also impaired performance for odor-only memory retrieval in multisensory trained flies (Fig. 2e-f), despite having no effect on memory retrieval in flies trained with only odors (Extended Data Fig. 2a,d).This unexpected result led us to hypothesize that multisensory learning might expand the KC representation of odors to include 'visual' γd KCs.
To directly test for learned odor-evoked responses in γd KCs after multisensory training, we expressed the voltage sensor UAS-ASAP2f 17 in γd KCs and performed two-photon functional imaging (Fig. 3a-d,g-j).We trained flies in the T-maze using a multisensory (color+odor), unisensory (odor), or unpaired protocol (where sugar was presented 2 min after color+odor).6 h after training, flies were imaged for their CS− and CS+ odor responses.Recordings of γd KC axons were made in the terminal γ5 compartment of the MB horizontal lobe, since key sugar rewarding dopaminergic neurons (DANs) drive learning-relevant presynaptic depression of KC-MBON synapses in the γ4 and γ5 compartments [18][19][20] .For comparison we also imaged γd KC responses in the more proximal γ1 compartment, which houses the presynaptic field of DANs that provide aversive teaching signals [21][22][23][24][25][26] .Odor presentation was previously shown to evoke slow inhibitory responses in γd 13 and αβp KCs 27 of naïve flies.
Consistent with these findings, presentation of the CS-odor evoked hyperpolarisation of γd KCs in both the γ1 and γ5 compartments, regardless of the training protocol (Fig. 3a-d and Extended Data Fig. 3a-b; light purple trace).However, after multisensory training the CS+ odor produced a significant depolarization of γd KC axons in the γ5 compartment (Fig. 3a, dark purple trace).CS+ responses in γ1 appeared to be less inhibited than those of the CS-(Fig.3b, although the responses were statistically indistinguishable) perhaps as a result of the hunger-state dependent control of the γ1pedc DAN [28][29][30] .Importantly, the multisensory training driven sign reversal of the γd KC odor response in γ5 did not occur following unisensory odor only (Fig. 3c; dark purple trace trace) or unpaired training (Extended Data Fig. 3a-b; dark purple trace).In these cases both the CS+ and CS-odors evoked hyperpolarization of γd KCs in the γ1 and γ5 compartments.These results indicate that dopaminergic reward teaching signals broaden CS+ odor-evoked excitation within the γ KC ensemble by recruiting the γ5 segments of γd KC axons to become odor activated.This larger odor memory engram provides a mechanism for how odor memory performance is enhanced following multisensory training, and explains why odor memory retrieval in this context acquires a requirement for γd KC output (Fig. 3e).
If DANs direct the recruitment of γd axons to become odor activated, a learning event that engages DANs innervating a distinct compartment should confer odor-responsiveness onto a different downstream segment of γd axons.Aversive memory formation requires dopamine release from neurons including the PPL1-γ1pedc DANs that innervate the most proximal γ lobe compartment -γ1 31 .We first confirmed that multisensory odor and color aversive (electric shock punishment) training produced a memory enhancement for both the combined and individual cues, similar to that observed after multisensory appetitive training (Fig. 3f, Extended Data Fig. 3c-h).We next used 2-photon imaging of the ASAP2f voltage sensor to test whether γd axons from γ1 onwards gained CS+ odor activation after multisensory aversive learning.Presentation of the CS-odor evoked hyperpolarisation of γd KCs in both the γ1 and γ5 compartments, regardless of the training protocol (Fig. 3g-j and Extended Data Fig. 3 i-j; light purple trace).In contrast, exposure to the CS+ odor after multisensory aversive training produced brief excitation of γd KCs in the γ1 and γ5 compartments.Recordings of odorevoked responses in γd KCs following appetitive and aversive multisensory learning are therefore consistent with the location of dopaminergic teaching signals determining the portion of γd axons that becomes activated by the CS+ odor.Whereas aversive learning makes all of the γd axon segments downstream of γ1 excitable by the CS+ odor (Fig. 3f), reward learning primarily alters the CS+ excitation within the terminal γ4 and γ5 segments (Fig. 3e).
An expansion of the CS+ odor representation into a particular segment of the γd axons after multisensory training might be expected to facilitate subsequent learning with the same odor, if the next dopamine teaching signal intersects the expanded KC representation.We tested this notion by sequentially training flies with either an aversive (dopamine in γ1) or appetitive (dopamine in γ4 and γ5) multisensory protocol followed by unisensory odor-reward or odorpunishment learning (Fig 3k-l), respectively.Prior multisensory aversive training significantly enhanced subsequent odor-reward learning (Fig. 3k).However, no performance enhancement was apparent if aversive odor learning followed multisensory appetitive learning (Fig. 3l).Therefore the multisensory training-dependent expansion of the CS+ odor representation can be included into the next CS+ odor memory engram if the appropriate γd axon segments have become CS+ odor-activated.
The anatomy of the MB network suggests two possible ways for odor responsiveness to be conferred to γd KC axons: via KC-KC synapses or neurons positioned to bridge the different KC streams.We queried the anatomical feasibility of these routes using the complete MB connectome of the adult female fly 'hemibrain' electron microscope (EM) volume 32,33 .Although most (562 of 590) γm KCs make synapses with γd KCs, the number and placement of these connections does not support every γd KC to receive γm input in every γ lobe compartment.
In addition, KC-KC connections have also been reported to suppress activity in neighboring KCs 34 .We next studied the fine anatomy of the γ lobe innervation of the potentially excitatory serotonergic Dorsal Paired Medial (DPM) neuron in the hemibrain EM volume.DPM neurons send separate branches that densely innervate the vertical and horizontal lobes and distal peduncle of the MB, where they are both pre-and post-synaptic to the KCs [35][36][37][38] .Analyzing the ultrastructure of DPM neuron projections in the horizontal γ lobe revealed two branches within the γ1 compartment and other ventral and dorsal branches passing through the γ2 -γ5 compartments (Fig. 4a).The positions of DPM neuron synapses onto γd KCs follow the γd KC axon bundle as it winds around the γ lobe from ventral in γ1 to dorsal in the γ5 compartment (Fig. 4a).
Annotating a dendrogram of DPM neuron neurites (Fig. 4b) with γ lobe compartment boundaries (based on DAN connectivity), synapses from γm KCs and those to γd KCs, showed that unique branches of the DPM neuron can provide compartment-specific microcircuit bridges between γm and γd KCs.DPM neurons can also bridge γd to γm KC connectivity (Extended Data Fig. 4g).The large GABAergic anterior paired lateral (APL) 37,39 neuron was found to make synapses along the DPM branches in the γ lobe (Fig. 4b and Extended Data Fig. 4h), suggesting DPM bridging can be regulated by inhibition.Importantly, the APL neuron receives many DAN inputs within each compartment and can therefore also be regulated with region specificity (Extended Data Fig. 4h), to potentially release specific DPM branches from APL inhibition.
We challenged this putative microcircuit bridge model by independently manipulating the APL and DPM neurons.Expression in APL neurons of the DopR2 dopamine receptor has been linked to aversive learning 40 and transciptional profiling suggests APL neurons also express the DopEcR receptor 41 .Both of these dopamine receptors are known to be inhibitory 42,43 .We therefore used tubP-GAL80 ts 44 to temporally-restrict transgenic RNAi in APL neurons to test for a role of these receptors in multisensory learning.Knocking down Dop2R in adult APL neurons abolished the multisensory enhancement of odor memory performance (Fig. 4c).A mild defect was also observed for odor memory following olfactory appetitive conditioning, however the difference was only significant to one of the controls (Fig. 4d).In contrast, DopEcR RNAi did not produce a defect in either experiment (Extended Data Fig. 4a-b).These results are consistent with reinforcing dopamine inhibiting APL neurons to allow the recruitment of γd KCs into the olfactory memory engram during multisensory learning.
We next tested for a role of the serotonergic DPM neurons using expression of UAS-Shi ts1 .
Temporally restricting neurotransmission from DPM neurons either during the acquisition (Fig. 4e) or retrieval (Fig. 4f) phases significantly impaired the multisensory training enhancement of odor memory performance.These same manipulations had no effect on odor memory performance after unisensory olfactory learning (Extended Data Fig. 4e), consistent with prior work 45 .We propose that DPM neuron output is required for learning because it binds together simultaneously active KC streams, whereas DPM neuron output during memory retrieval provides the connection for odor-driven γm KCs to activate the relevant γd KCs.Serotonin (5hydroxytryptamine, 5-HT) can exert excitatory effects through 5-HT2A and 5-HT7 type receptors 46 .We therefore used RNAi to knockdown these 5HT receptors in γd KCs and tested olfactory memory following multisensory training.Reducing 5-HT2A but not 5-HT7 or 5-HT2B receptor expression impaired memory performance (Fig. 4g).Moreover, bath application of 5-HT evoked a clear excitatory response in γd KCs expressing ASAP2f in naïve flies (Fig. 4h-i).
Taken with prior knowledge that learning specifically increases responsiveness of DPM neurons to the CS+ odor 36 , these anatomical, genetic and physiological data lead us to conclude that reinforcer-evoked compartment specific dopamine releases APL-mediated inhibition which permits the formation of an odor-color specific DPM microcircuit bridge between the relevant γm and γd KCs (Fig. 4j).

Discussion
Our study describes a precise neural mechanism in Drosophila through which multisensory learning improves subsequent memory performance, even for individual sensory cues.We found that a single training trial with visual cues could only generate robust memory performance in flies if they were combined with odors during training, similar to visual rhythm perception learning in humans, which requires accompanying auditory information 47 .We show that multisensory learning binds together information from temporally contingent odors and colors within the axons of mushroom body γ KCs, via the serotonergic DPM neurons.

Importantly, this learning-driven binding converts axons of visually (presumably color)
selective KCs to also become responsive to the temporally contingent trained odor.While predominantly olfactory or visual dendritic input respectively defines γm KCs as being olfactory and γd as being visual, our recordings show that segments of their axons in the MB lobes become multimodal after multisensory learning.This result suggests that γ KCs are a likely substrate where other temporally contingent sensory information can be integrated with that of explicit sensory cues 48,49 .
Although our experiments focused on odor activated γm KCs recruiting color γd KCs via DPM microcircuits, the observed behavioral enhancement of visual memory following multisensory learning and the connectivity of DPM neurons suggests that the reverse situation is also likely to apply.In so doing, multisensory learning links the KCs that are responsive to each temporally contingent sensory cue and expands the representation of each cue into that of the other.This cross-modal expansion allows the multisensory experience to be efficiently retrieved by the combined cues and by each individually.As a result, trained flies can evoke a memory of a visual experience with the learned odor, an effective form of synesthesia.These findings provide a neural mechanism through which the fly achieves a conceptual equivalent of hippocampus-dependent pattern completion in mammals, where partial cues are able to retrieve a more complete memory representation 50 .Interestingly, human patients with schizophrenia and autism exhibit deficits in multisensory integration 51,52 and these conditions have also been linked to serotonergic dysfunction and 5-HT2A receptor activity 53,54 .Our work here suggests that inappropriate routing of multisensory percepts may contribute to these conditions.Moreover, the excitatory 5-HT2A receptors that mediate multisensory binding, are the major targets of hallucinogenic drugs 55 .
Behavioral experiments.Male flies from the GAL4 lines were crossed to UAS-Shi ts1 virgin females, except for experiments involving c708a-GAL4, where UAS-Shi ts1 males were crossed to c708a-GAL4 virgin females.For heterozygous controls, GAL4 or UAS-Shi All behavioral experiments were conducted using a standard T-maze that was modified to allow simultaneous delivery of odor and color stimuli.The T-maze which is made from translucent plastic was covered in opaque blackout film to minimise interference between the visual stimuli when they were used in parallel.Odors were 4-methylcyclohexanol (MCH) and 3-octanol (OCT) diluted in mineral oil (at ≈1:10 -3 dilution).Colors were provided by lightemitting diodes (LEDs); green LEDs with wavelength 425 +/-10nm (ProLight Opto, PM2E-3LGE-SD) and blue LEDs with wavelength 465 +/-10nm (ProLight Opto, PM2B-3LDE-SD).
Four LEDs were assembled in a circuit built onto a heat sink and were mounted securely on top of the odor delivery tubes.The intensities of the LEDs were adjusted so that naïve flies showed no phototactic preference between the illuminated T-maze arms.Visual stimuli were presented in the same manner and same intensity for both training and testing.For appetitive experiments the testing tubes were lined with filter paper and for aversive experiments the testing tubes were lined with non-electrified shock grids.Experiments were performed in an environmental chamber set to the desired temperature and 55-65% relative humidity.Flies were handled prior to training and testing under overhead red light.
Appetitive conditioning was performed essentially as previously described 10 .Briefly, flies were exposed for 2 min to stimuli Y (YOdor and/or YColor) without reinforcement in a tube with dry filter paper (the Conditioned Stimulus minus, CS-), 30 s of clean air, then 2 min with stimuli X (XOdor and/or XColor) presented with 5.8 M sucrose dried on a filter paper (the Conditioned Stimulus plus, CS+).For aversive olfactory conditioning 11,12 , flies recieved 1 min exposure to stimuli X (XOdor and/or XColor) paired with twelve 90 V electric shocks at 5 s intervals (CS+), 45 s of clean air, followed by 1 min exposure to stimuli Y (YOdor and/or YColor) without reinforcement (CS-).Electric shocks were delivered using a Grass S48 Square Pulse Stimulator (Grass Technology).Shock grids are those previously described 13 and consist of interleaved copper fingers printed on translucent Mylar film which allows color stimuli to penetrate.
Memory performance was assessed by testing flies for their preference between the CS-and the CS+ odors and/or colors for 2 min.Odor testing was performed in darkness.Performance Indices were calculated as the number of flies in the CS+ arm minus the number in the CSarm, divided by the total number of flies.For all behavioral experiments, a single sample, or n, represents the average Performance Index from two independent groups of flies trained with the reciprocal odor/color combinations as CS+ and CS-.
Six behavioral protocols were used: 1) Visual Learning (V), colors (XColor and YColor) were used as CS+ and CS-.
2) Olfactory Learning (O), odors (XOdor and YOdor) were used as CS+ and CS-.Two-Photon Voltage Imaging.All flies were raised at 25 ˚C and 3-8 day-old male and female flies were used in all experiments.Imaging experiments were performed essentially as described previously [14][15][16] .In brief, flies were trained in the T-maze setup using either Olfactory learning (protocol 2 above), a Congruent multisensory protocol (3 above) or an Unpaired training protocol.In Unpaired training, flies were exposed to the combined odor and visual stimuli (XOdor+XColor and YOdor+YColor combination), but the shock or sugar was presented alone 2 min before or after the CS+, respectively.After training, flies were kept in darkness until recording.Just before recording, flies were briefly immobilized on ice and mounted in a custom-made chamber allowing free movement of the antennae and legs.The head capsule was opened under room temperature carbogenated (95% O2, 5% CO2) buffer solution and the fly, in the recording chamber, was placed under a Two-Photon microscope (Scientifica).
Flies were subjected to a constant air stream, carrying vapor from mineral oil solvent (air).To emulate the testing phase, the flies were sequentially exposed to CS+ and CS-odor, each for 5 s, interspersed by 30 sec of air.As in the behavior experiments the odors were MCH and OCT (diluted in mineral oil at ≈1:10 -3 ), and they were used reciprocally as CS+ and CS-.One hemisphere of the brain was randomly selected to image the axons of KCs.Any flies that did not respond to one of the two presented odors were excluded from further analyses.Each n corresponds to a recording from a single fly.
Fluorescence was excited using ~140 fs pulses, 80 MHz repetition rate, centered on 910 nm generated by a Ti-Sapphire laser (Chameleon Ultra II, Coherent).Images of 256 x 256 pixels were acquired at 5.92 Hz, controlled by ScanImage 3.8 software 17 .Odors were delivered using a custom-designed system 18 .
For acute 5-HT application, we used a perfusion pump system (Fisher Scientific US 14-284-201) to continuously deliver saline at a rate of ~0.043 mL/sec.5-HT was applied in the presence of 1 µM tetrodotoxin (TTX) to block voltage-gated sodium channels and propagation of action potentials that could result in indirect excitation.To examine the effects of serotonin on γd KC membrane voltage, baseline fluorescence was recorded for 5 min before switching to a solution containing 100 μM serotonin hydrochloride (Sigma Aldrich, Cat# H9523) for an additional 5 min of recording.Washout was performed by changing the solution back to saline.The time of application and concentration of serotonin used is comparable to recent physiological studies applying exogenous serotonin to the Drosophila brain [19][20][21][22] .Due to perfusion tubing length and dead volume, the perfusion switch took approximately 70 s to reach the tissue.
For analysis, two-photon fluorescence images were manually segmented using Fiji 23 , using a custom-made code including an image stabilizer plugin 24 .Movement of the animals was small enough such that images did not require registration.For subsequent quantitative analyses, custom Fiji and MATLAB scripts were used.The baseline fluorescence, F0, was defined for each stimulus response as the mean fluorescence F from 2 s before and up to the point of odor presentation (or 30 s after the start of the recordings for 5-HT treatments).F/F0 accordingly describes the fluorescence relative to this baseline.For the imaging of KCs, the area under the curve (AUC) was measured as the integral of -ΔF/F0 during the 5 s odor stimulation.The total number of flies, for each training protocol, came from 3 different training sessions.
For 5-HT treatments we defined the "pre-" treatment as the average -ΔF/F0 value for 300 s prior to the 5-HT delivery, the 5-HT application was the average -ΔF/F0 for 300 s during 5-HT delivery and the "washout-" treatment as the average -ΔF/F0 for 300 s from the offset of drug delivery.Traces were smoothed over 5 s by a moving average filter.The total number of flies were acquired from 3 different imaging sessions.
ASAP2f data are presented as -ΔF/F0 to correct the inverse relation between sensor fluorescence and membrane voltage.
Statistical Analysis.Statistical analyses were performed in GraphPad Prism.All behavioral data were analyzed with an unpaired t-test, Mann Whitney test or a one-way ANOVA followed by a posthoc Tukey's or Dunnet's multiple comparisons test.No statistical methods were used to predetermine sample size.For the imaging experiments odor-evoked responses were compared by a paired t-test for normally distributed data, otherwise a Wilcoxon matched-pairs signed rank test was used for non-Gaussian distributed data.Normality was tested using the D'Agostino & Pearson normality test.For imaging data, a method for outlier identification was run for each dataset (ROUT method), which is based on the False Discovery Rate (FDR).The FDR was set to the highest Q value possible (10%).In datasets in which potential outliers were identified, statistical analyses were performed by removing all odor-evoked responses for those flies.The analyses with or without the outliers were not different, so we decided to maintain and present the complete datasets, which may contain potential outliers.
Custom scripts based on navis.plot_flat()were used to generate dendrograms of DPM and APL neurons with twigs of Strahler order ≤1 pruned.MB compartment boundaries were defined by connectivity to DANs of the respective compartments.Branches outside the γ-lobe were downsized manually to increase visibility of γ-lobe compartments.Synapses are filtered by in_volume() and displayed on branches with Strahler order >1.Connectivity statistics are based on unpruned neurons and synapses between neurons were obtained with R based natverse:: neuprint_get_synapses (https://natverse.org) 25 scripts and processed with custom scripts in Python.Asterisks denote significant difference (P < 0.05).Data presented as mean ± SEM.Individual data points displayed as dots.See Extended Data Fig. 2 for temperature controls.Asterisks denote significant difference (P < 0.05).h-i.Left top, imaging plane in the γ1 (h) and γ5 (i) region of γd KCs.Left bottom, for all experiments, a baseline recording in saline (300 s; not shown) was followed by perfusion with 100 μM serotonin (5-HT; 300 s) and by washout in saline (300 s).Average traces for bath application and washout in γ1 (h) and γ5 (i) are shown.
ts1 flies were crossed to WT flies.All flies were raised at 25 ˚C, except where noted below for manipulation of RNAi expression.Populations of 2-8 day-old flies were used in all experiments.For appetitive conditioning experiments, 80-100 flies were placed in a 25 ml vial containing 1% agar (as a water source) and a 20 × 60 mm piece of filter paper for 19-22 h before training and were kept starved for the entire experiment, except when assaying 24 h memory where flies were fed for 30 min after training then returned to starvation vials until testing.For aversive conditioning experiments, 80-100 flies were placed in a vial containing standard food and a piece of filter paper for 14-22 h before behavioral experiments.For experiments involving neuronal blocking with UAS-Shi ts1 , a schematic of the timing of the temperature shifting is provided in each figure.For Shi ts1 experiments, flies were transferred to restrictive 33 ˚C 30 min prior to training and/or testing.For RNAi experiments involving tubP-GAL80 ts ; VT43924-GAL4.2,flies were raised at 18 ˚C and shifted to 29 ˚C after eclosion to induce RNAi expression for 3 d before the behavioral experiments.The flies remained at 29 ˚C for the duration of the experiments.

3 )
Congruent protocol (C), odors and colors were combined (XColor + XOdor and YColor + YOdor) as CS+ and CS-.The same odor + color combinations were used during training and testing.4) Incongruent protocol (I) odor and color stimulus contingencies were switched between training (XColor + YOdor and YColor + XOdor) and testing (XColor + XOdor and YColor + YOdor).The V and O protocols are unisensory, whereas C and I are multisensory.5) Olfactory retrieval (OR), flies were trained as in the Congruent protocol (C), but only odors (XOdor and YOdor) were presented as the choice at test.6) Visual retrieval (VR), flies were trained as in the Congruent protocol (C), but only colors (XColor and YColor) were presented as the choice at test.The sequential learning experiments depicted in Figures 3k and 3l used aversive or appetitive Congruent multisensory training (C) followed by unisensory appetitive or aversive Olfactory learning (O) then testing using olfactory retrieval (OR).

Figure 1 .
Figure 1.Multisensory learning enhances memory performance.a. Left, Apparatus for multisensory training and testing.Right, training and testing timeline.b.Schematic of experimental protocols.Green and blue squares represent colors, light and dark grey squares represent 3-OCT and 4-MCH odors.Visual learning (V): colors were used as CS+ and CS-.Olfactory learning (O): odors were used as CS+ and CS-.Congruent protocol (C): pairs of colors and odors were combined as CS+ and CS-.Same color + odor combinations were used for training and testing.Incongruent protocol (I): colors and odors were combined as CS+ and CS-but color + odor combinations were switched between training and testing.Olfactory retrieval (OR): color and odor combinations were used for training but only odors for testing.Visual retrieval (VR): color and odor combinations were used for training but only colors for testing.c and d.Top, training and testing timelines.Bottom, immediate (c) and 6 h (d) memory performance for (V), (O), (C) and (I) protocols.Flies did not learn with colors alone (V).Performance was significantly increased when colors and odors were presented together.Congruent multisensory memory was significantly different than regular olfactory memory (O) and memory in the incongruent protocol (I) at all time points.At 6 h (d), memory performance for incongruent protocol (I) was significantly less than olfactory memory (O).e and f.Top, training and testing timelines.Bottom, multisensory training with colors + odors significantly enhanced immediate (e) and 6 h (f) memory for each individual modality.Performance retrieved with only odor (OR) or only colors (VR) was significantly greater than following unisensory olfactory (O) or visual learning (V).Asterisks denote significant differences (P < 0.05).Data presented as mean ± standard error of mean (SEM).Individual data points displayed as dots.

Figure 2 .
Figure 2. Enhanced performance following multisensory learning requires visuallyresponsive γd and αβp Kenyon Cells.a. Top, schematic of γd KCs.Bottom, training and testing timeline with temperature shifting (dashed line).Right, blocking output of γd KCs during testing using MB607B-GAL4; UAS-Shi ts1 impairs 6 h memory performance in the congruent protocol.b.Blocking γd KC output during testing improved 6 h memory performance in the incongruent protocol.c.Top, schematic of αβp KCs.Bottom, training and testing timeline with temperature shifting (dashed line).Right, blocking αβp KC output during testing using c708a-GAL4; UAS-Shi ts1 reduces 6 h congruent memory performance.d.Blocking αβp KC output during testing improved 6 h memory performance in the incongruent protocol.e-f.Blocking γd KC output during testing impaired Olfactory Retrieval of multisensory memory (e), whereas blocking αβp KCs had no effect (f).

Figure 3 .
Figure 3. γd Kenyon Cells become odor-activated after multisensory learning.a. Left, appetitive multisensory (color + odor) training and imaging (odor) timeline.Imaging plane in the γ5 region of γd KC axons.Middle, traces of CS+ and CS-odor-evoked activity.The γ5 region of γd KC axons showed an excitatory response to the CS+ odor (a decrease in fluorescence increases the -ΔF/F0, of the ASAP2f voltage sensor).The same γ5 axons were inhibited by the CS-odor (decrease in −ΔF/F0).Right, quantification of odor-evoked responses.b.Left, appetitive multisensory (color+odor) training and imaging (odor) timeline.Imaging plane in the γ1 region of γd KC axons.Middle, traces of CS+ and CS-odor-evoked activity.Excitatory responses to CS+ odor were not observed in γ1 and the CS-odor elicited inhibition (decrease in −ΔF/F0).Right, quantification of odor-evoked responses.c-d.Left, appetitive unisensory (odor) training and imaging (odor) timelines.Imaging plane in the γ1 (c) and γ5 (d) regions of γd KCs.Middle, traces of CS+ and CS-odor-evoked activity.Both γ1 and γ5 regions show inhibition in response to CS+ and CS-odors (decrease in −ΔF/F0).Right, quantification of odor-evoked responses.For all traces and quantification, CS+ data correspond to average responses in which 50% of trials used MCH as CS+ and the others OCT was CS+.Same applies for CS-data.Odor-evoked activity traces show mean (solid line) with SEM (shadow).Black line underneath traces marks 5 s odor exposure.Asterisks denote significant difference between averaged CS+ and CS-responses (P < 0.05).CS+ and CSresponses for each individual fly are connected by a dashed line.e. MB model for appetitive multisensory color + odor training followed by unisensory odor testing.γ KCs project through the γ1-γ5 compartments of the horizontal γ lobe.γ main [γm] KCs receive olfactory input while γ dorsal [γd] are visually driven.Appetitive training (left) engages rewarding DANs (green) that innervate the γ4 and γ5 compartments.Dopamine encodes learning by directing depression of synapses between stimulus-activated KCs and avoidance-directing Mushroom Body Output Neurons (MBONs; not illustrated) 9 .Dopaminergic signalling during multisensory learning also binds γm and γd KC activity in the γ4-5 compartments.During subsequent unisensory testing (right) odor excites specific γm KCs (thick grey), which in turn activate γd axons in the γ4-5 compartments (grey dashed lines to yellow).The additional KC activity increases memory expression.f.MB model for aversive multisensory training followed by odor testing.Aversive multisensory training (left) engages punishment DANs (red) that depress synapses betweenγm and γd KCs and approach-directing MBONs56,57 (not shown) in proximal γ1 and γ2 compartments, while also binding γd and γm KC activity in these compartments.Unisensory odor testing (right) excites specific γm KCs which also activates γd axons from γ1 forward.g.

Figure 4 .
Figure 4. DPM and APL neurons mediate multisensory stimulus binding.a. Left, 3D representation of the MB (light grey) with Dorsal Paired Medial (DPM) neuron neurites of the γ lobes (teal).DPM trifurcates (asterisk) into dorsal (dark teal) and ventral branches (light teal) and a γ1 compartment branch.Neurites are shaded by Strahler order and twigs with Strahler order <1 are pruned.Right, detailed view of the γ lobe with γ1-γ5 compartment borders defined by dashed lines.DPM presynapses to γd KCs (yellow spheres) co-localize on the dorsal branch in the γ5 compartment.Input synapses from γm KCs (grey spheres) to DPM are located throughout the ventral and dorsal branches.In γ5, 450 of 585 γm KCs make synapses with the DPM neuron dorsal branch, where 89 of 98 γd KCs also receive DPM input.Anterior Paired Lateral (APL) neuron inputs (magenta spheres) localize along both DPM branches.b.A 2-dimensional dendrogram projection of DPM neurites (shades of teal -see Extended Data Figure 4 g for details).The γ1 compartment is marked and γ2-5 compartments are split between dorsal and ventral DPM neuron branches.Inputs from γm KCs (grey spheres) and outputs to γd KCs (yellow spheres) co-localize on compartment-specific branches.Inhibitory inputs from APL (magenta spheres) are distributed Figure 1 Figure 4 a