7
$\begingroup$

Many of the variables in the data I use on a daily basis have blank fields, some of which, have meaning (ex. A blank response for a variable dealing with the ratio of satisfactory accounts to toal accounts, thus the individual does not have any accounts if they do not have a response in this column, whereas a response of 0 means the individual has no satisfactory accounts).

Currently, these records do not get included into logistic regression analyses as they have missing values for one or more fields. Is there a way to include these records into a logistic regression model?

I am aware that I can assign these blank fields with a value that is not in the range of the data (ex. if we go back to the above ratio variable, we could use 9999 or -1 as these values are not included in the range of a ratio variable (0 to 1)). I am just curious to know if there is a more appropriate way of going about this. Any help is greatly appreciated! Thanks!

Improve this question
$\endgroup$
1
  • 5
    $\begingroup$ Missings that really mean 0 (or something else) can be recoded as such, although that might have implications for what model is sensible. Missings that are recoded arbitrarily will usually lead to nonsensical results with any model. Otherwise the only option is imputation. $\endgroup$
    – Nick Cox
    Aug 12, 2013 at 17:26

1 Answer 1

Reset to default
6
$\begingroup$

In general, dealing with missing input values is always problematic. To my best knowledge, none of the existing methods can deal with it without introducing some bias to the model, so you have to consider this during your research. There are at least few possible options:

  • ignore data with missing values (which I do believe you do now), which is the "safest" option, but can lead to insufficient data being left to train a good model
  • fill missing values with some statistical analysis of the data - for example:
    • mean value of the particular feature/dimension (for real valued variables)
    • median value of the particular feature/dimension (for categorical ones)
  • train a separate model to predict a missing value, e.g. let's imagine data in $X^k$, and each of the dimensions can have missing inputs, then you can create $k$ models $M_i$, each for predicting the $i$th dimension using the rest of them, so $M_i : X^{k-1} \rightarrow X$, and you use it to preprocess your data
  • use some generative model, that can fill missing values by itself, one possibility is a Restricted Boltzmann Machine

As was previously stated, each method introduces some bias to the analysis (which has been proven in many papers, for many models), but it can also help you build a better model: everything depends on your data.

EDIT (after clarification)

A missing value of some $i$th feature/dimension $f_i \in X$ is lack of observation/knowledge about what particular value $x\in X$ does it have. One can imagine a situation where we are asking people to fill out a multi-page survey, and after getting all the data it turns out we do not have one of the person's pages. We do not know what was his/her response, but we are quite sure there was one. On the other hand a person could give as a blank question (without an answer) or write something like "I will not answer this question", which is not missing information; in fact this is as informative as selecting one of the predefined boxes. In such a scenario we simply have a categorical feature, $f'_i \in X \cup \{ \emptyset \}$. We can either express it as a multi-valued feature, or encode it in unary form by replacing $f'_i$ with $|X|+1$ new binary features $f''_{ij}$ for each $j\in X \cup \{ \emptyset \}$ such that $f''_{ij} = 1 \iff f'_i = j$. Choice between these methods is model- and data-dependent.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ I'm not so sure that option one ('ignore data with missing values') is considered 'safest' anymore -- it seems to be generally agreed that imputation (options 2 or 3 in your list) are preferred methods in terms of lower bias from the modelled outputs. Multiple imputation (often based on a regression model like option 3) is the current preferred option in biostatisticsm at any rate, and there are plenty of materials on-site here on this issue. $\endgroup$
    – James Stanley
    Aug 12, 2013 at 20:41
  • 1
    $\begingroup$ In my opinion, the fact, that it is often used, does not mean that it is 'safe'. It simply means, that the advantages of having more information are bigger then a risk of biasing the whole research. As far as I know, machine learning community still finds the ignoring of missing values the most conceptually correct method (as it gives a clear view of what is actually being modelled, when you use some kind of imputation - often on many levels - it is really hard to tell what is being modelled at the end). But as I said - everything depends on your data. $\endgroup$
    – lejlot
    Aug 13, 2013 at 5:16
  • $\begingroup$ With the data that I use, the missing data has importance, the values of missing all have the same value. I am simply wondering how to better assign these variables a value (they NEED to be included), if possible, to include them into logistic regression analyses. Almost every record typically has a missing value in one (or many more) of the 1000+ variable fields that I work with. Any ideas with this added information? Thanks again for all of your helpful input thus far! $\endgroup$
    – Matt Reichenbach
    Aug 13, 2013 at 12:11
  • $\begingroup$ If your "missing" value has a separate meaning then having a 0/1 value, then it is not a missing value, it is simply a feature(dimension, variable) with 3 possible values (categorical variable), lets say 0,1,2. There at least two possible ways of representing them - either you choose three distinct values, or create 3 binary features $x'_1,x'_2,x'_3$ where $x'_1 = 1 \iff x=i$ which could be better some models. To sum up - these are not missing values, this is simply a third possible value. $\endgroup$
    – lejlot
    Aug 13, 2013 at 13:29
  • 3
    $\begingroup$ @lejlot: I'm comfortable with having a difference of opinion: My main concern was to note that your first statement isn't a universally held opinion. Just to clarify my earlier comment with respect to your reply: when I said "Multiple imputation...is the current preferred option in biostatistics" my intended meaning was "the current academic position in biostatistics, based on theory and formal investigation, is that [properly applied] MI methods are less prone to bias than complete case analysis" rather than "everyone does it." $\endgroup$
    – James Stanley
    Aug 13, 2013 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.