2
$\begingroup$

I have a longitudinal dataset with records from 22902 individuals. The outcome is a continuous variable and has no missing values. However, one of the predictors (from a total of 8) has some missing values. This predictor refers to the patient's smoking status (Yes or No), and I have no record of such status for 9430 individuals.

Since this variable does not change over time, I think the smoking status was accessed upon the first medical appointment and kept that way; in cases where the information is unavailable, the patient must not have disclosed their smoking status when first evaluated. So I don't think it is related to the outcome.

How to handle this? Suppose I remove the cases where the values are missing. In that case, I lose information on 9430 patients. So I thought about creating a third category: "missing" or "unknown", and carrying on with the analysis. But I'm not sure that would be the correct way of handling this.

I appreciate any help you can provide. Thanks!

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ I would use mice stats.stackexchange.com/questions/421545/… complete case analysis of single imputation is usually pretty bad $\endgroup$
    – rep_ho
    Commented Oct 7, 2022 at 16:35
  • $\begingroup$ @rep_ho I tried to use mice. But it appears I'm doing something wrong. My categorical covariates are all time-invariant. However, mice is returning me different categories for the same subject; the smoking status for a given subject is "yes", and at some point in time, it changes to "no", and vice-versa. $\endgroup$
    – adrimsvieira
    Commented Oct 7, 2022 at 21:34
  • $\begingroup$ @rep_ho thanks; I was able to implement the mice function successfully!!! After I reshaped my dataset, I got the same categories for the same subjects. $\endgroup$
    – adrimsvieira
    Commented Oct 8, 2022 at 2:36

2 Answers 2

Reset to default
1
$\begingroup$

Using a separate category for missing covariate data is called the "Indicator Method." Stef van Buuren discusses that in Section 1.3.7 of Flexible Imputation of Missing Data (FIMD). In some restricted circumstances that can be OK, but he warns:

the method ... generally fails in observational data. It has been shown that the method can yield severely biased regression estimates, even under MCAR [missing completely at random] and for low amounts of missing data...

If you knew that your smoking status was MCAR then you might be OK with the listwise deletion of cases without smoking status; see Section 2.7 of FIMD. You would, however, be throwing away a lot of data and have coefficient standard errors on the order of 30% higher than what you would have with the entire data set.

Follow the advice of @rep_ho in a comment, and use multiple imputation to get multiple copies of the data with different estimates of the smoking status, and put the information from their models together as explained by van Buuren, who developed the mice package in R for that purpose. As the missing data is restricted to smoking status, that should mean that the imputation-associated error is mostly restricted to the smoking coefficient while you take advantage of the full data set for all predictors.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I tried to use mice. But it appears I'm doing something wrong. My categorical covariates are all time-invariant. mice is returning me different categories for the same subject; the smoking status for a given subject is "yes", and at some point in time, it changes to "no", and vice-versa. Any idea why? I don't want to use complete cases analysis since I would be throwing away more than 40% of my data. That's why I first considered the NA values as a third category for my covariate. $\endgroup$
    – adrimsvieira
    Commented Oct 7, 2022 at 21:44
  • $\begingroup$ @adrimsvieira The mice() function imputes values for all data rows. Section 6.4 on Derived Variables might provide some clues. Or you could post-process the smoking imputations to make sure they are all the same for an individual within an imputation set, say by choosing the majority vote among rows for each individual in each imputation set and applying that to all rows for that individual in that imputation set. $\endgroup$
    – EdM
    Commented Oct 7, 2022 at 23:34
  • $\begingroup$ @adrimsvieira Section 11 on Longitudinal Data might be even more on point, particularly if the nature of your data allow for expression of values in a "wide" format with one (wide) row for each individual. Then you could set up the wide format so that it only included one column for each of the time-constant predictors. $\endgroup$
    – EdM
    Commented Oct 7, 2022 at 23:41
  • $\begingroup$ thank you so much! This is really useful. Now I was able to use the mice function for my data. $\endgroup$
    – adrimsvieira
    Commented Oct 8, 2022 at 2:31
0
$\begingroup$

In this case, your idea seems correct of treating "not disclosed" as a third value of the categorical variable.

Are any of the available descriptors correlated with smoking? Blood pressure for example? If so, then you should take care about the choice of model - Random Forest would work better than Logistic Regression in face of the partial correlation.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I'm not sure I understood your question 😅 My outcome is a continuous variable. I want to use the linear mixed-effects model to fit the data. Besides the smoking status, I have factors like having diabetes, hypertension, etc as other predictors... $\endgroup$
    – adrimsvieira
    Commented Oct 7, 2022 at 21:54
  • $\begingroup$ Among the patients who do not disclose whether they smoke, a higher fraction of hypertensive patients are really smokers than non-hypertensive ones (at least, that seems likely). It follows that hypertension is a stronger predictor of your outcome variable among no-reply patients than among admitted smokers or declared non-smokers. A model that assumes that the predictors make independent contributions to the outcome will not handle this well. Random Forest is therefore a good choice of model in this case. $\endgroup$
    – chrishmorris
    Commented Oct 10, 2022 at 8:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.