0
$\begingroup$

How should one approach the following problem?

Suppose an object has an unknown binary attribute X in {0, 1} (for example it is only possible to be either green = 1 or red = 0), and has a known attribute set {A = a, B = b, C = c} (for example weight = a, height = b, length = c). We have the following information:

60% of all objects seen previously that had A=a among their attribute values had X = 0.

And 10% of all objects seen previously with B=b had X=0.

And 45% of all objects with C=c had X = 0.

We have no information regarding combinations of attributes of past objects. What is the probability of the object having X = 0?

$\endgroup$
0
$\begingroup$

There is no single answer to this problem unless you make assumptions about the combination of attributes of past objects. For instance, consider a simpler, 2-attribute case with $A, B \in \{0, 1\}$ with equal independent probability of each combination. Suppose we know that $P(X = 0 \mid A = 0) = P(X = 0 \mid B = 0) = P(X = 0 \mid A = 1) = P(X = 0 \mid B = 1) = 0.5$. This knowledge is consistent with either of the two dramatically different probability distributions:

$$\begin{array}{ccc} P(X=0) & A=0 & A=1 \\ B=0 & 0.0 & 1.0 \\ B=1 & 1.0 & 0.0\end{array}$$ or $$\begin{array}{ccc} P(X=0) & A=0 & A=1 \\ B=0 & 1.0 & 0.0 \\ B=1 & 0.0 & 1.0\end{array}$$

As this hopefully demonstrates, you need to make an assumption about the combinations of attributes of past objects. What assumption is reasonable depends on your data. For instance, one common assumption is that features are independent of each other given the value of $X$. In this case, you would end up with a naive Bayes classifier. These have been shown to work reasonably well in some cases even if the assumption of independence is violated, but of course you should validate that classifier on your own data to see how well it works for you.

$\endgroup$
  • $\begingroup$ Thanks for the answer. How can one implement Naive Bayes without a training set though? $\endgroup$ – Audrey May 5 '15 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.