1
$\begingroup$

Please check the image for reference How to choose the prior probability of each feature .

Should we use the same prior for every feature or different features use different prior.

for example P(Color = Blue | Stolen = Yes ) = (0 + 1(1/3)) / 5 + 1 , where I used the smoothing parameter (alpha) = 1 and p = 1/3 as color can have 3 values i.e Red Blue and Yellow. So I am a little confused here , should I consider Blue also here or should I just consider Red and Yellow

Also for the other features should I use the same p = 1/3 or for them it should be p=1/2 as they only have two values .

Kindly check the image enter image description here

$\endgroup$
2
  • 1
    $\begingroup$ Could you give us more details on what exactly were you asked to do and what exactly is the Laplace smoothing formula you are referring to? The usual definition of Laplace smoothing has only one parameter. $\endgroup$
    – Tim
    May 20, 2021 at 7:30
  • $\begingroup$ Given the Attributes that are< Color = Blue , Type = Sports , Origin= Domestic >, predict using Naive Bayes that whether the car can be stolen or not. The formula for Laplace smoothing is almost the same as you provide, however in the numerator, a prior P is multiplied to the alpha. Here is the image for further reference $\endgroup$
    – Skywalker
    May 20, 2021 at 15:31

1 Answer 1

1
$\begingroup$

The more standard definition of Laplace smoothing is defined as adding a fixed quantity $\alpha$ to each count

$$ \hat p_i = \frac{n_i+\alpha}{\sum_{j=1}^K n_j+\alpha} = \frac{n_i+\alpha}{(\sum_{j=1}^K n_j) + K\alpha} $$

where $\alpha$'s can be thought as "pseudocounts", i.e. with $\alpha=1$ you assume that for each of the categories you observed $+1$ sample like this, with higher values leading to more smoothing and lower values to less smoothing.

Splitting $\alpha$ to the two parameters $m$ and $P$ so $mP=\alpha$, to me seem unnecessarily complicated. Still, if you remember that $mP$ is a "pseudocount" added to each observed count, it may be easier to grasp an intuition on what the reasonable values could be.

$\endgroup$
2
  • $\begingroup$ so K here is the no of values in that feature ? Assume training set contains 10 positive examples, and the feature “Size” has 3 values, but 1 value (size=medium) is not observed in 𝐷: 4: small 0: medium 6: large so K here will be 3 ? P(small | positive) = (4 + 1) / (10 + 3) = 0.394 P(medium | positive) = (0 + 1) / (10 + 3) = 0.03 $\endgroup$
    – Skywalker
    Jun 16, 2021 at 7:58
  • $\begingroup$ @Skywalker correct, $K$ is the number of categories, so as in your example, if there are three possible sizes, $K=3$. $\endgroup$
    – Tim
    Jun 16, 2021 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.