I have data from a humidity sensor and would like to estimate the confidence interval of its mean value. However, since the time stamps are "too close together" the data-points are correlated. Therefore, I have to come up with a clever idea to handle the autocorrelation. The following solutions came to my mind:

  1. Removing neighbouring data-points until they are no longer autocorrelated. This is valid, however, most probably this does not yield the most accurate result.
  2. First estimating the autocorrelation function $\rho$ and the variance $\sigma^2$, then using

$Var[\bar{X}_i] = \frac{\sigma^2}{N} + (1 - \frac{1}{N}) \rho \cdot \sigma^2$

which is valid if

$E[X_i]=0$, $Var[X_i]=\sigma^2$, and $Cov[X_i, X_j] = \rho \sigma^2$

However there must be more accurate methods using time series analysis, aren't there? This is such a common problem, but I couldn't find an answer.

  • $\begingroup$ These repeated data points might not be carrying any more information than the others, so my two cents suggestion would be for either removing or aggregating these points. This might help you. $\endgroup$ – Lucas Farias Feb 12 at 1:03

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.