4
$\begingroup$

Consider the dataset:

Outlook Temperature Humidity Play Golf?
Overcast Cool Low Yes
Sunny Hot Low Yes
Rainy Cool High No
Sunny Hot High No
Rainy Cool Low Yes

There are 3 possible values for the feature "Outlook":

  • Overcast
  • Sunny
  • Rainy

P(Yes) = 3/5 and P(No) = 2/5.

The probability of Overcast given No is P(Overcast | No) = 0. I do know to solve this issue, we use Laplace Smoothing with the formula, α/(N + α⋅k). In this case if α = 1, 0 + 1/(2 + 1⋅k). What I'm unsure about is what does k here means.

So my questions are:

  1. Does k refer to the number of features (Outlook, Temperature, Humidity) or the number of possible values for Outlook?
  2. Do we apply Laplace Smoothing for the feature Outlook or to all other features too?
  3. Do we normally check if conditional probability is 0 before applying Laplace Smoothing or do we just apply it straight?
$\endgroup$

1 Answer 1

1
$\begingroup$

$k$ refers to the possible values for Outlook, because you'll need the sum of all Outlook probabilities be $1$. It's up to you to apply this to all features or not. The smoothing is useful for remedying zero probability cases, but it's also for applying regularization. The degree/strength of regularization depends on the value of $\alpha$ and can be chosen by validation. So, zero probability is not the sole driver behind it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.