Lets assume that our binary class has five features x1,x2,x3,x4,x5 but the incoming test vector(x1,x2,x3,x4) has only four features .. How would we handle it in the Naive bayes classifier .. Do we take put in the values of x5 feature separately and take the mean or is there some other way to handle this?

  • $\begingroup$ Why is the missing feature missing? $\endgroup$ – Tim Apr 20 '18 at 11:38
  • $\begingroup$ @Tim Its just not there ..I just want to know what would the solution ..What would be the right way to go about it? $\endgroup$ – Zeist Apr 20 '18 at 11:50
  • 1
    $\begingroup$ Have a look at the other dozens of similar questions tagged [data-imputation] and [missing-values]... this has been likely solved already. Also, @Tim's question is very relevant: why is the value missing? Is it missing in all testing data? Without knownin anything about the nature of your missing values it is impossible to tell you what to do with them... $\endgroup$ – Jan Kukacka Apr 20 '18 at 11:56

If you know nothing about why the missing feature is missing, then imputing it with anything is risky. When imputing the missing values based on what you observed in your data, you make an assumption that the missing values would be somehow similar to the values you observed. If the mechanism that lead to missingness is unknown, then it can be the case that the values are not missing at random and there is some kind of dependence between the fact that the values are missing and the rest of your data (e.g. people who didn't like your product didn't answer the question, so their other answers about the product will differ as well). So in this case, if the feature is missing, it would be the best idea to ignore it.

Native Bayes algorithm uses the pairwise conditional empirical distributions between the features and target variable and makes the native assumption about independence. So it doesn't care about the dependence between the features. This means that to make predictions with one feature dropped, you don't have to recalculate anything, just ignore the feature when making predictions.

More formally, Native Bayes uses the Bayes theorem to estimate

$$ p(C_k|x_1,\dots,x_n) \propto p(C_k) \prod_{i=1}^n p(x_i|C_k) $$

and take

$$ \hat{y} = \underset{k \in \{1, \dots, K\}}{\operatorname{argmax}} p(C_k|x_1,\dots,x_n) $$

With $n-1$ features, just take the features you have and apply the formula. If would possibly give you different predictions then if you had the feature, but you don't have it and nothing better can be done.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.