4
$\begingroup$

I have a dataset of 100000 samples. The samples represent the failure time of an electronic component that fails after a given number of "shocks", whatever shock means. We know that these systems fail with a random distribution that is represented by a Gamma distribution. Finally, we are told that measures are affected by simple white Gaussian noise. How can I verify that this dataset is IID?

I know that a "demonstration" of indipendency is the autocorrelation function but what do I use to verify that are identically distributed?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
4
$\begingroup$

So, you have a very long timeseries, that is observations of some variable (here failure time) in time sequence. You ask two questions: 1) are the observations independent? 2) do they all have the same marginal distribution?

As for the first question, timeseries are seldom independent. As you say, the autocorrelation function can be useful to investigate this. I will concentrate on the second question: Are all the marginal distributions identical? In that form, in that generality, the question cannot be answered. All statistical analysis needs some assumptions, and assumptions about equal distribution are among the most basic ones. But, to even talk about a distribution, we need an assumption that some individual properties of objects (here electronic components that may fail) are unimportant. So, we are not really interested in each component individually, only in some aggregate properties, and the distribution of failure times is one such. But, still it might be the case that there are some differences in these aggregate properties: components might come from different sources (brands, production lines, ...) so you might aggregate data according to such factors and ask: Is the distribution of failure time the same in each group? Draw a histogram for each group, and compare.

Another possibility is some slow drift with time in the distributions, it might be caused by some drift in properties of the production process, or of the measurement process. So, divide the data into some (10 or more) nonoverlapping intervals, make a histogram for each, and compare them. You might also use some more formal analysis such as changepoint analysis. See this list: https://stats.stackexchange.com/search?q=changepoint+analysis for some relevant CV posts.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ +1 In addition to histograms (which have been criticized as having arbitrary aspects to their appearance, and often hide important characteristics of distributions), one ordinarily would consider boxplots (and their variants) and perhaps probability plots for such exploratory comparisons of distributions. $\endgroup$
    – whuber ♦
    Commented Apr 23, 2015 at 15:11
  • $\begingroup$ Yes, but here it is mentioned 100000 samples so histograms should do well ... $\endgroup$
    – kjetil b halvorsen ♦
    Commented Apr 23, 2015 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.