2
$\begingroup$

It is well known that the expected value of a function can be approximated with i.i.d. samples:

$$ E_X[f(X)] = \frac{1}{n}\sum_{i=1}^n f(x_i),\quad x_i\sim_{i.i.d.} X $$

What methods exist to approximate the expected value of the function when only "dependent" samples are available? What does it actually mean to have i.i.d. samples in this context?

On a related question, how would one go about checking that a sample $\{x_1,x_2,\ldots,x_n\}$ is drawn i.i.d. from some random variable $X$?

$\endgroup$
  • $\begingroup$ MCMC and SMC algorithms rely on dependent samples and still approximate the expectation with converging and sometimes unbiased estimators. $\endgroup$ – Xi'an Jan 19 at 7:47
2
$\begingroup$

I believe that by "approximated" you mean "estimated".

1) As long as you assume that the $x_i$ all come from the same distribution, the sample mean of the $f(x_i)$ (as given in your posting) is at least unbiased. I don't think that without information about the specific dependence structure you can do better than that, although dependence may make this estimator much less precise.

2) There are some tests for independence. The best known one is probably the runs test; also one could test for autoregression parameters to be zero in an autoregressive time series model. However, without specifying an alternative, I'd look at the data in time order and/or by any variable that could induce dependence and see whether any deviations from independence are visible. Note that independence can never be verified, such tests only can falsify it, and none of them is perfect. The first thing to do is always to think about how the data were obtained, and if any specific source of dependence comes to mind.

$\endgroup$
  • $\begingroup$ Thank you @Lewian, and sorry for the delay to reply. In my use case, the samples are spatial and so we know that they are dependent. We could possibly index/order the samples via a multidimensional index. $\endgroup$ – juliohm Jan 24 at 0:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.