Detecting Signal from Data with Noise

Xianyao Chen
First Institute of Oceanography
Laboratory of Data Analysis and Applications

A priori assumptions on the background noise and the associated statistical methods, used for the detection of signal from noisy data by rejecting a noise null hypothesis, mask the possibility that the detected oscillations are generated from the stochastic processes of another kind. In order to detect signals from the data with multi-types of noise, an adaptive null hypothesis is developed with the finding that true physical signals in a well-sampled time series cannot be destroyed or eliminated by re-sampling the time series with fractional sampling rates through linear interpolation. Therefore, the significance of potential signals can be tested by checking whether the signals persist in the true time-frequency spectral representation during re-sampling. The test of this hypothesis is based on the general characteristics of noise as revealed by Empirical Mode Decomposition, an adaptive data analysis method without linear or stationary assumptions, and without any pre-definition of the background noise. Applications of this method to synthetic time series, financial data, and climatic time series such as solar spots number, and sea-surface temperature time series illustrate its power in identifying characteristics of background noise without any a prior knowledge.

Presentation (PowerPoint File)

Back to Adaptive Data Analysis and Sparsity