Suppose we have two datasets $A$ and $B$ with size $S_A$ and $S_B$. Instances of datasets have multiple features. Consider the specific features $X$ and suppose we are given $PDF$ of this feature in each dataset. Now we take a random instance and observe that $X=1.2$ for this instance and we want to guess which dataset this instance belongs to, i.e : comparing $p(From_A|X=1.2)$ and $p(From_B|X=1.2)$.
However, $p(X=1.2) = 0$ since $X$ is a continuous variable and this leads to a problem because nominator and denominator of the following fraction will be $0$ :

$p(From_A|X=1.2) = \dfrac{p(X=1.2|From_A)*p(From_A)}{p(X=1.2)}$

I think discussing about $P(A|B)$ when $B$ is impossible, is meaningless. So how can we guess what dataset this instance belongs to?

  • 1
    $\begingroup$ There is something mixed up here. A "data set" is usually finite, and so X cannot be continuous on this data set. Please explain the process by which you take a "random instance". $\endgroup$ – user3697176 Sep 12 '15 at 16:37
  • $\begingroup$ $X$ is a feature of an instance and it can be continuous. we have 250 instances from dataset $A$ and 250 instances from dataset $B$. Now we are given an instance that value of $X$ in this instance is $X=1.2$. can you guess which dataset this instance was from? $\endgroup$ – Mohammad Sep 12 '15 at 16:54
  • $\begingroup$ I can always guess :). Based on the information in your comment, I would expect my guess to be right half of the time :) $\endgroup$ – user3697176 Sep 12 '15 at 16:58
  • $\begingroup$ Why not work with X in [1.2, 1.2+eps] instead and take limits as eps -> 0? $\endgroup$ – user3697176 Sep 12 '15 at 16:59
  • $\begingroup$ @user3697176 : I had guessed the $0.5$ probability but it means that this probability is completely independent of $PDF$ of $X$ in each dataset which seems strange... $\endgroup$ – Mohammad Sep 12 '15 at 17:24

As creosote said that Wikipedia article is very helpful. You are correct that the denominator is zero and your intuition to take limits is a great one. What you did not realize is that the first factor of your numerator is also zero. So really you have a quotient of densities which is perfectly reasonable to manipulate. The denominator will require you to mix your two densities for A and B.

As a last note, as another commentator said. It is reasonable to say that the feature is continuously distributed in the population you sampled from, but in your dataset it is certainly discrete.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.