1
$\begingroup$

I have some data that was collected by different sensors. I compute some statistics over this data, like the mean. I would like to get an idea of how much my results are sensitive to the particular sensors that I'm using. The reason is that more sensors may be added in the future and I would like to get some confidence that the results will hold then.

I have first thought of using a bootstrap approach, but instead of sampling from the data uniformly, I sample the sensors with replacement and form each bootstrap sample from the data that was collected by the selected sensors (if one sensor is selected twice, the related data is inserted twice into the sample). After that, I compute the 95% confidence interval for the mean of the resulting data samples.

I am not at all certain that this approach yields valid confidence intervals or even valid mean estimates because each sensor may have collected a different amount of data from the others and, since they are imperfect, they may have a different bias from one another, and thus may produce measurement that are not exactly identically distributed across sensors.

I've been looking for a method that could help me answer my original question. I've come across "block bootstrap", but I haven't yet found any resource to help me understand if it'd solve my problem. Any such resource or pointers to other methods that could help me would be appreciated.

$\endgroup$
  • 1
    $\begingroup$ Yours is a cluster bootstrap, with sensors being clusters. It takes care of a correlation of observations within the cluster/sensor. Makes good sense. $\endgroup$ – StasK Oct 17 '16 at 17:08
1
$\begingroup$

There are cases for which the mean (A.K.A. expected) value does not exist, or for which the mean value is not the best measure of location of a population, for example for the Cauchy distribution or certain values of the Pareto distribution. Thus, the first step in examining a distribution, provided there is enough data to do so and the sample size is as a minimum around 40 would be to plot a histogram and do a search for the best distribution to fit it. When one identifies a distribution type, then one also identifies the measures of location, dispersion, and other measures analogous to moments, even when those moments do not exist. For example, for the Cauchy distribution the median is a measure of location and the mean is undefined. Moreover, for this, FWHM exists as a measure of dispersion, and the standard deviation is undefined.

Once measurements that characterize the data are chosen (as opposed to guessed at), their significance becomes clearer. Bootstrap, in turn, becomes a method of finding those characterizing measurements. And, if bootstrap is used to compute arbitrary parameters without first characterizing the bootstrap distribution, the results will be arbitrary.

$\endgroup$
  • $\begingroup$ Interesting remark. I was aware that some distributions did not have a mean, but I hadn't thought that computing the sample mean may not make sense. So, you are suggesting that one should always make assumptions on the underlying distribution, even when computing a simple mean, is that right? I'm not sure however how that would help me evaluate the sensitivity of my results to the particular sensors I'm using. i.e. Would my results hold with other sensors or are they driven by a few sensors in my hands that are skewed? $\endgroup$ – Guillaume Chérel Oct 18 '16 at 7:58
  • $\begingroup$ Some of the models are so bad, even if they appear to have high r-values, that non-parametric measurements make more sense to use than the parametric ones. Consider a faulty sensor. If it includes occasional wild values in the readings, a mean value would be much more sensitive to that noise than a median. Plotting a histogram and fitting with many candidate distributions might show a mixture "best" distribution of 'good' and 'bad' readings and allow one to segregate the population of readings, from the population of errors. $\endgroup$ – Carl Oct 18 '16 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.