How to interpret MCAR (missing completely at random) any papers that I can read?

Question

When dealing with missing data and Little's missing completely at random test, it's widely considered that if the test has a significance level of P>0.05 the data can be considered as MCAR.

But, I can't find a single paper that confirms that interpretation. All the papers just say that their values are higher than 0.05 so their data set is good and can reject the null hypothesis etc etc but they never confirm that where they got that "0.05" value. I was searching for hours. I found the following paper of Little (1988) http://www.jstor.org/stable/2290157

I couldn't find anything in that paper that it states about significance (P) of greater than 0.05 can be considered as data completely missing at random. Am I looking for the wrong key words? can anyone help me? Are there any research papers that confirm this?

UPDATE: In a nutshell, I'm asking, I need evidence by research papers of "P>0.05 can be considered as the data is missing completely at random"

Answer 1

You're right to be skeptical. The whole business of interpreting a significance test's failure to reject a null hypothesis as evidence for the null hypothesis is illegitimate. It just means that on the particular metric your test considered, the data you have isn't extremely unlikely under the null hypothesis.

Furthermore, missingness completely at random is a strong assumption and should not be made merely on the basis of quantitative analyses of the data at hand (after all, the question is what's missing from what you have), but also on the qualitative question of how you think the data came to be missing.

Answer 2

Paul Allison's discussion of MCAR in his Sage monograph, Missing Data, has one of the clearest explanations of MCAR in the literature. For instance on p. 3, regarding a test of the MCAR mechanism hypothesis, he writes:

The data on Y are said to be missing completely at random if the probability of missing data on Y is unrelated to the value of Y itself or to the values of any other variables in the data set...it's easy to test this by dividing the sample into those who (...are vs those who are not missing...) and then testing for a mean difference. If there are, in fact, no systematic differences on the fully observed variables between those with data present and those with missing data, then the data are said to be observed at random. On the other hand, just because the data passes this test does not mean that the MCAR assumption is satisfied. Still there must be no relationship between missingness on a particular variable and the values of that variable...

So, to @Kodiologist's point, the assumptions for MCAR are strong and a simple test does not, in and of itself, satisfy those assumptions. They merely lend evidence in its support.

Answer 3

UPDATE: In a nutshell, I'm asking, I need evidence by research papers of "P>0.05 can be considered as the data is missing completely at random"

MCAR, MAR and MNAR characterize the causal mechanism responsible for missing data.

If you were investigating some other question not related to missing data, and you got a p-value above 0.05, it would be wrong to say "P>0.05 proves that there is no effect".

For the same reason, P>0.05 does not prove that the missing data are MCAR, because p-values don't prove the absence (or presence) of effects, but indicate whether a model is consistent with some data.

Say you get a p-value of 0.04 using Little's MCAR test. That means, given that your missing data is MCAR, you would get the type of missing data pattern that you have in your data (or a more extreme pattern) 4 percent of time (in the context of repeated studies).

P = 0.38 => 38% of the time, you'd see this type of missing data pattern (or a more extreme one). So if you have good reasons to believe your missing data is MCAR, you might go ahead as if it were true, even though it might not be.