I have data from a humidity sensor and would like to estimate the confidence interval of its mean value. However, since the time stamps are "too close together" the data-points are correlated. Therefore, I have to come up with a clever idea to handle the autocorrelation. The following solutions came to my mind:

  1. Removing neighbouring data-points until they are no longer autocorrelated. This is valid, however, most probably this does not yield the most accurate result.
  2. First estimating the autocorrelation function $\rho$ and the variance $\sigma^2$, then using

$Var[\bar{X}_i] = \frac{\sigma^2}{N} + (1 - \frac{1}{N}) \rho \cdot \sigma^2$

which is valid if

$E[X_i]=0$, $Var[X_i]=\sigma^2$, and $Cov[X_i, X_j] = \rho \sigma^2$

However there must be more accurate methods using time series analysis, aren't there? This is such a common problem, but I couldn't find an answer.

  • $\begingroup$ These repeated data points might not be carrying any more information than the others, so my two cents suggestion would be for either removing or aggregating these points. This might help you. $\endgroup$ Feb 12, 2019 at 1:03


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.