Sensory cue combination, and in particular multisensory cue combination has served as the classic example for optimal probabilistic inference operating in the nervous system. Numerous behavioral studies have shown that humans integrate the information from two or more cues as predicted by a maximum likelihood estimator. However, in these studies the discrepancy among the signals have been small, and paradigms have been used in which the sensory signals were assumed to originate from a single source, and therefore, the different cues were completely fused. While complete integration may be a common scenario in within-modality cue combination (such as combination of various depth cues in vision), it constitutes only a fraction of scenarios in crossmodal sensory combination. The process of cue combination across modalities has to solve an additional problem, that of when to integrate the sensory signals (in addition to the problem of how to integrate them).
The nervous system, at any instant, is confronted with the critical problem of estimating which sensory signals have been caused by the same source and should be integrated, and which have been caused by different sources and should be segregated. Behavioral studies show that depending on the degree of discrepancy in space, time, and structure, the crossmodal signals may be fully integrated, partially affect each other, or processed independently (segregated). I will present a Bayesian model that can coherently address both questions of when and how to combine signals. I will also show behavioral experiments that were conducted to test this model. Results of these experiments show that human processing of multisensory information is highly consistent with the Bayesian ideal observer. Therefore, Bayesian inference seems to provide a unifying account for the entire spectrum of cue combination, ranging from no integration, to partial interactions, to complete fusion.
Copyright. All Rights Reserved.