# How to deal with gaps/NaNs in time series data when using Matlab for autocorrelation and neural networks?

I have a time series of measurements (heights-one dimensional series). In the observation period, the measurement process went down for some time points. So the resulting data is a vector with NaNs where there were gaps in the data. Using MATLAB, this is causing me a problem when computing the autocorrelation (autocorr) and applying neural networks (nnstart).

How should these Gaps/NaNs be dealt with? Should I just remove these from the vector? Or replace their entry with an interpolated value? (if so how in MATLAB)

-

There are some algorithms which are immune to missing values, so the preferred solution is to look for them (for instance R's acf for autocorrelation).

In general, the way to go is to either just discard data with missing observations (might be very painful) or just to impute their values -- mean of neighbors might be enough for smooth series and small gaps, but there are is of course of plethora of other more powerful methods, using splines, random/most frequent values, imputation from models, etc.

-
The acf with missing value is calculated in the normal way, but missing values are 'skipped' in the sums (that is, acf formula for a given lag looks like a sum divided by a sum, in each of those sums the missing values can be skipped). This is not the same as removing the missing values from the original data. The problem with matlab is it doesn't skip NaN, and including that in the calculation turns everything into NaN. –  Zero Apr 17 at 3:39

I would not touch the data at all. Use this for autocorrelation with NaNs:

http://www.mathworks.com/matlabcentral/fileexchange/43840-autocorrelation-and-partial-autocorrelation-with-nans/content/nanautocorr.m

"not touch the data" means not to remove any data or time-step or replace with 0 or the mean, it would compromise the information about the specific-time-lag linear dependence. I would also avoid simulating the values in the gaps, if you are interested in the "SAMPLE" autocorrelation, anyway even the best simulation technique will not add any more information about the autocorrelation, being based on the data themselves. I partially recoded the matlab (link above) autocorrelation and partial autocorrelation functions to deal with NaNs: any data couples including NaNs is excluded from the computation. This is done for each lag. It worked for me. Any suggestion is well accepted.

-
Welcome @Fabio: Could you please give more explanation to what you mean with "not touch the data"? Do you mean not to remove anything? It would also help to introduce the content you linked to and explain why this helps the OP. –  Momo Oct 10 at 10:22
Hello Momo, thanks for the comment. "not touch the data" means not to remove any data or time-step or replace with 0 or the mean, it would compromise the information about the specific-time-lag linear dependence. I partially recoded the matlab (link above) autocorrelation and partial autocorrelation functions to deal with NaNs: any data couples including NaNs is excluded from the computation. This is done for each lag. It worked for me. Any suggestion is well accepted. –  Fabio Oct 21 at 17:47