0
$\begingroup$

I have a data-set which has n number of features containing both categorical and continuous values. its a binary classification problem. The data set has many missing values. I am interested in imputing the values. For features such as amount/price I could use regression, Query : I could predict the values using other features(which have missing values), is this the right way?is the predicted value good/correct enough? Can I calculate the correlation between the variables for the whole data-set with missing values?

$\endgroup$
1
  • $\begingroup$ I suggest reading and doing more research on this. You need to establish the mechanism of missingness. This is a good place to start. $\endgroup$
    – Zhubarb
    Apr 25, 2019 at 14:23

1 Answer 1

0
$\begingroup$

There is a multitude of ways on how to treat missing values.

The most straight-forward but brutal way is to drop all data points where values are missing. However, this is not necesserily the best way, especially if the percentage of data with missing values is large. Other than that it depends on the feature itself and you should consider what makes sense according to the feature's meaning. Other simple options include:

  • Set a constant for the missing values.
  • Set the missing value to a statistic of the feature (mean, median, mode)

Edit: As mentioned in the comments these methods might be too simple and lead to inferior results. A more sophisticated and well-performing approach is multiple-imputing.

$\endgroup$
2
  • 1
    $\begingroup$ Why the downvote? At least leave feedback. $\endgroup$
    – amaik
    Apr 25, 2019 at 13:21
  • 5
    $\begingroup$ Not my downvote, but you should know that there is extensive discussion of imputation on this site under the tags data-imputation and multiple-imputation. Your point about the brutality of dropping cases with missing values is correct, but multiple imputation is generally accepted as the most reliable approach, as it incorporates the uncertainties introduced by the process for filling in the missing data. $\endgroup$
    – EdM
    Apr 25, 2019 at 15:11

Not the answer you're looking for? Browse other questions tagged or ask your own question.