The feeling of familiarity can be triggered by stimuli from all sensory modalities, suggesting a multimodal nature of its neural bases. In the present experiment, we investigated this hypothesis by studying the neural bases of familiarity processing of odors and music. In particular, we focused on familiarity referring to the participants' life experience. Items were classified as familiar or unfamiliar based on participants' individual responses, and activation patterns evoked by familiar items were compared with those evoked by unfamiliar items. For the feeling of familiarity, a bimodal activation pattern was observed in the left hemisphere, specifically the superior and inferior frontal gyri, the precuneus, the angular gyrus, the parahippocampal gyrus, and the hippocampus. Together with previously reported data on verbal items, visual items, and auditory items other than music, this outcome suggests a multimodal neural system of the feeling of familiarity. The feeling of unfamiliarity was related to a smaller bimodal activation pattern mainly located in the right insula and likely related to the detection of novelty.