My Little's MCAR (missing completely at random) test on 74 items and 151 cases revealed
chi-square = 27.120, DF = 1974, and sig. = 1.000.
Can I conclude that the data were missing completely at random since the p-value is not significant?
And, is there any issue with my p-value because I thought it was impossible to get $p = 0$ or $p=1$?
A large p-value (> 0.05) indicates weak evidence against the null hypothesis, so you fail to reject the null hypothesis, in this case the null hypothesis is that the data is MCAR, no patterns exists in the missing data.
Proving the existence of MAR data is difficult but you can try if data is related between them. The package Hmisc in R has some graphical tools to see the relationship between each variable. Another idea could be to do a logistic regression with the outcome being missing vs no missing for each variable and see if any other predictor is associated with the missingness of the variable.
As a final note, I would say to think about your data and the definition of MCAR. Do you think it's plausible for the data to be MCAR? If so, then I would say there is evidence that the data is MCAR.
Hope this helps.
tl;dr: Little's test is probably not well-powered enough to detect missingness. You're probably testing for the wrong kind of missingness and won't be able to learn about the kind of missingness you really care about. The things you would do to handle data that are MAR or covariate-dependent-MCAR are things you should just do anyway.
First, it's important to understand what MCAR, MAR and MNAR mean, technically. If your data are MCAR, this means that whether an observation of your outcome variable (y) is missing does not depend on either the observed or unobserved values of y, nor on any covariates upon which y depends. As an equation, this gives you:
$$ \mathbb{P}(\mathbf{\text{R}}_{i} = {\widetilde{\mathbf{\text{r}}}}_{i}\,|\,\mathbf{\text{Y}}_{i,(1)} = {\widetilde{\mathbf{\text{y}}}}_{i,(1)},\,\mathbf{\text{Y}}_{i,(0)} = \mathbf{\text{y}}_{i,(0)},\,\mathbf{\text{X}}_{i},\,\mathbf{\text{Z}}_{i},\,\mathbf{\alpha}) = \mathbb{P}(\mathbf{\text{R}}_{i} = {\widetilde{\mathbf{\text{r}}}}_{i}\,|\,\mathbf{\text{Z}}_{i},\,\mathbf{\alpha}). $$
In other words, the probability of an observation of y being missing ($\mathbf{\text{R}}_{i} = {\widetilde{\mathbf{\text{r}}}}_{i}$) depends only on some covariates that do not predict y (called $\mathbf{\text{Z}}$) and the coefficients that relate $\mathbf{\text{Z}}$ to y, called $\alpha$.
The case in which missingness is dependent on covariates $\mathbf{\text{X}}$ is termed "covariate dependent missingness" and is just a special case of MCAR. Importantly, in this case, $\mathbf{\text{X}}$ is a set of variables y is dependent on. Little's test can tell you whether these variables are related to missingness on y, but only if you are adequately powered to detect such an association. If you're not well powered, you may mistakenly conclude they are not related. So, what would be your recourse if Little's test comes out significant? You would then include those $\mathbf{\text{X}}$ in your model of y somehow. But since y is dependent on $\mathbf{\text{X}}$, they should be part of your model anyway, or else you risk omitted variable bias!
Little's test, then, is mostly useless: if your data are MCAR, you should include $\mathbf{\text{X}}$ to avoid omitted variable bias. If missingness depends on $\mathbf{\text{X}}$, you should do the same. If Little's test shows that other variables unrelated to y, i.e., $\mathbf{\text{Z}}$, are associated with missingness, you don't need to include them because the definition of MCAR allows missingness to be dependent on $\mathbf{\text{Z}}$. Note again, that you're likely to be underpowered to detect missingness with Little's test anyway.
So what about the case of missing at random, or MAR? This is the case when missingness on y is dependent on the values of y that you have observed. Following the above notation:
$$ \begin{matrix} {\mathbb{P}(\mathbf{\text{R}}_{i} = {\widetilde{\mathbf{\text{r}}}}_{i}\,|\,\mathbf{\text{Y}}_{i,(1)} = {\widetilde{\mathbf{\text{y}}}}_{i,(1)},\,\mathbf{\text{Y}}_{i,(0)} = \mathbf{\text{y}}_{i,(0)},\,\mathbf{\text{X}}_{i},\,\mathbf{\text{Z}}_{i},\,\mathbf{\alpha}) =} \\ {\mathbb{P}(\mathbf{\text{R}}_{i} = {\widetilde{\mathbf{\text{r}}}}_{i}\,|\,\mathbf{\text{Y}}_{i,(1)} = {\widetilde{\mathbf{\text{y}}}}_{i,(1)},\,\mathbf{\text{Y}}_{i,(0)} = \mathbf{\text{y}}_{i,(0)}^{\prime},\,\mathbf{\text{X}}_{i},\,\mathbf{\text{Z}}_{i},\,\mathbf{\alpha})} \\ \end{matrix}, $$
the important part of which is that on the left you have $\mathbf{\text{Y}}_{i,(0)} = \mathbf{\text{y}}_{i,(0)}$ and on the right you have $\mathbf{\text{Y}}_{i,(0)} = \mathbf{\text{y}}_{i,(0)}^{\prime}$, which just means that the equality holds whether the missing values of y would have taken on the values $\mathbf{\text{y}}_{i,(0)}$ or $\mathbf{\text{y}}_{i,(0)}^{\prime}$.
This situation doesn't make (at least to me) much sense outside of a longitudinal data context. In a situation where your cross-sectional observations are supposed to be statistically independent, it's not clear to me how an observed value on y for one person could tell you anything about the probability that another person's y observation would be missing. In longitudinal data, this situation arises because someone's observed y at time t-1 could be predictive of whether y at time t is missing.
Is Little's test useful here? Maybe. If it's significant, you should include all observed y in a maximum likelihood model if possible. If it's not significant, (and well powered enough that you don't risk a false negative), you could simply remove all cases that are missing at least one y. But you'll have better statistical power for your model of y if you just include all available data anyway. Again, your modeling decisions are the same either way, so why bother with Little's test?
Finally, if your data are missing not at random (MNAR), missingness depends on unobserved values of y and so are definitionally not statistically detectable.
For more in-depth discussion and several examples, see
Matta, T. H., Flournoy, J. C., & Byrne, M. L. (2018). Making an unknown unknown a known unknown: Missing data in longitudinal neuroimaging studies. Developmental cognitive neuroscience, 33, 83-98. 10.1016/j.dcn.2017.10.001
As far as I know, you can look at both right or left tail of chi squared test. Having p-value of exactly 1 it is possible to say (with some caution) that your data could be artificially generated and "too random". So it could be an issue with your p-value.
(here is the answer I am referring to)
Another issue - chi squared can be considered as a sum of squares of normally distributed RVs. If you use test with such test statistic and overestimated your variances for normal RVs - you will essentially get what you've got.
