Theory
Our model is constrained to the stationary and correlated noise. The stationarity is quantized by Augmented Dickey-Fuller (ADF) test which checks whether we are able to reject the null hypothesis (signal is non-stationary) based on the current confidence level. In our project, the built-in function adftest from MATLAB was used. The correlation of the noise was determined from the autocorrelation of the noise.
To extract signals buried in noise, we used a variation of the standard pth-order Autoregressive (AR) model for stationary processes. Before getting into the version we used, we first introduce the standard Autoregressive model[2]:
1
Here x't represents the predicted tth data point of the time series signal, xt represents the tth data point of the actual signal, and ak is the kth Autoregressive coefficient. In this way, each data point of the signal is modeled as the weighted sum of the previous p number of data points immediately preceding it.
The model we used is similar to the standard model, but instead of predicting the tth data point of the signal by the weighted sum of the p number of data points before it, we predict the t+T-1th data point by that summation, where T is some time lag. This is the Autoregressive model with time lag:
2
The significance of including time lag in the model will be explained later. This model was then applied to a segment of signal-free noise. Notice that we do not need any specific information about the signal, making a bold guess about which segment of the noise doesn't contain signal is already sufficient.
To visualize how this model will allow us to extract a signal buried in the noise, we created the following diagrams:
To determine the optimal Autoregressive coefficients, we used the method of least squares. This involves minimizing the mean square difference of the predicted noise and the actual noise:
3
where Tonset is the estimated onset time of the signal into the noise. As explained before, this estimation need not to be good as long as we can make sure that the segment of noise we are modelling is signal-free. To minimize Equation 3, we take its derivative with respect to each Autoregressive coefficient, and set each obtained equation to 0, thus minimizing S, i.e. minimizing the error of our model. We then obtain a system of p equations which we can solve for the p number of Autoregressive coefficients. To determine the optimal p, we find the Autoregressive coefficients for various values of p, and then calculate the Final Prediction Error, which satisfies Akaike’s Information Criterion [3], and is given by:
4
We take the p value for which this is minimized to be our optimal p (typically, from 30 to 80). And the corresponding model is then our optimal model.
The predication of the noise can then be obtained easily from the autoregressive model with optimal order and coefficients. The last step is to subtract this predicted noise from the original signal to obtain the error. One restriction that we have to bear in mind, however, is that this method assumes the stationarity of the sample process. This criteria cannot be fulfilled after the desired signal comes in. Our model should fail dramatically since the introduction of the signal breaks the stationarity. Consequently, an error peak can be expected time lag T after introducing the desired signal due to the lag in our model. The onset time is then estimated as time lag T before the presence of the large error peak in the error plot. The desired signal can be extracted in the range from the estimated onset time to the presence of the large error, because our prediction of the noise in this range is still valid as it is based on the data from time lag T before, when signal has not yet come in. Then by subtracting off the prediction noise, we should be left with the desired signal in the error plot.