I read few responses close to the question and was suggested in using t-test or chi-sq test. However, the pattern between variables can also involve more than 2 variables (e.g. data at x tend to be missing when data at y and z equals to "yes"). Is there any comprehensive package in Python/R to check the type of missingness in a tabular dataset?
$\begingroup$
$\endgroup$
4
-
1$\begingroup$ There are already several threads that answer this, please check them. TL;DR you can't really, because you would need to analyse the data that you are missing for that. $\endgroup$– TimCommented Jul 13, 2020 at 13:07
-
$\begingroup$ How about correlation matrix as a baseline technique? Turn the columns into missing/non-missing and then pearson correlation $\endgroup$– user291195Commented Jul 13, 2020 at 22:37
-
$\begingroup$ Correlation of the values you have with something does not tell you anything about correlation of the values you don't have with this thing, so I'm not sure if I understand what you mean? $\endgroup$– TimCommented Jul 13, 2020 at 22:43
-
$\begingroup$ I mean, we can turn the dataframe into binary of whether data is present or missing (assuming that the dataset that we have was collected in an unbiased manner). If the missingness in column A and column B is highly correlated, then it'll show that the missingness in col A or B could potentially be MNAR(?). Or something more advanced like multiple logistic regression? I think missingness can be different in different columns (e.g. one dataset can have different types missingness) $\endgroup$– user291195Commented Jul 14, 2020 at 1:13
Add a comment
|