# Questions tagged [laplace-smoothing]

Laplace smoothing (also known as additive smoothing) is a technique associated with a probability regularisation task. It ensures that certain improbable outcomes are still associated with a minimum probability of occurrence.

48 questions
Filter by
Sorted by
Tagged with
1 vote
41 views

### Computing heuristic from winrate and number of games

I need to sort big sets of players (of a MOBA) based on their performance represented by their - number of - games and their winrate on these games. I reduced the initial set of players based on their ...
10 views

### How to interpret Diffusion Maps for the iris dataset?

This might be a poor exercise but I'm trying to understand the methods of paper and if it makes sense to adapt my linear-based workflow with PCA to non-linear manifold methods; thought trying out ...
• 1,482
28 views

### How to replace empirical frequency predictions with Laplace estimates

I have frequency predictions for a discrete distribution: $p(x_1)=0$, $p(x_2)=0$, $p(x_3)=0.05$, $p(x_4)=0.95$ I need to smooth the distribution so I don't have zero values. I think the solution is ...
• 675
1 vote
65 views

### What is the influence of initial state in sequence generated from a markov chain?

For thousands of item, I have observations about their state (a letter) for 9 timestep. From that, I build a transition matrix (RotationMatrix by couting their ...
• 151
1 vote
62 views

### Impact of Laplace smoothing on likelihood in Naive Bayes

When 1 is added to word count in Laplace Smoothing in Naive Bayes, the new probabilities either increase or decrease as shown below. Though the problem of "zero" probability has been solved. ...
1 vote
29 views

### Laplace Smoothing in Naive Bayes [duplicate]

I'm reading up on Laplace Smoothing/Add-1 Smoothing in Naive Bayes and I'm given the formula $\frac{Count(Feature=Value) + α}{N + α\cdot k}$. In reference to the image above, if we have to classify ...
2k views

### What attributes does Laplace Smoothing apply on in Naive Bayes

Consider the dataset: Outlook Temperature Humidity Play Golf? Overcast Cool Low Yes Sunny Hot Low Yes Rainy Cool High No Sunny Hot High No Rainy Cool Low Yes There are 3 possible values for the ...
3k views

### How can Naive Bayes overfit the data?

I know that Laplace smoothing results in a high bias of Naive Bayes. If the value of the smoothing parameter (alpha) is large, then the probability distribution will be uniform for all the features. ...
1 vote
1k views

### How to compute KL-divergence when there are categories of zero counts?

I have two very large discrete frequency distributions (about 4 million items), and each contains many items with counts of 0. I want to calculate the KL divergence between them and use the empirical ...
1k views

### Calculating perplexity with smoothing techniques (NLP)

This question is about smoothed n-gram language models. When we use additive smoothing on the train set to determine the conditional probabilities, and calculate the perplexity of train data, where ...
• 23
627 views

### Naive Bayes - having trouble coming up with a case where Laplace smoothing changes the prediction

I'm thinking through the logic of Naive Bayes and encountered a brain teaser. I know that adding smoothing (alpha) to Naive Bayes can help to increase the accuracy of the model, which implies that it ...
• 123
325 views

### How should I handle Laplace smoothing in Naive Bayes in this example?

I have a toy dataset on animals, with 4 features and 2 possible classes (mammals vs non-mammals). I have summarized the dataset ...
• 111
1 vote
596 views

### Why do we need to apply Laplace smoothing to all the words in Naive Bayes for text classification?

I understood that we need to apply for Laplace smoothing to the words that are not present in our training data. But then why/what is the need to do Laplace smoothing for all the words (even the words ...
834 views

### How is Laplace Smoothing used in this example of Binary classification in Naive Bayes

I am following CS229 course by Andrew Ng. On this lecture note it talks about using Laplace smoothing to bypass situations of 0-probabilities. What does not make sense is the immediate jump to the ...
1 vote
3k views

### Hoes does laplace smoothing in Naive Bayes control high bias and high variance?

I'm trying to understand how laplace smoothing exactly helps to balance between overfitting and underfitting. I know that Laplace smoothing is used as a fail safe probability if there's a any ...
• 113
1 vote
2k views

### Global sensitivity of mean and variance in differential privacy?

Please explain me why global sensitivity of a mean or variance queries will be (b-a)/n and (b-a)^2/n where b is the upper ...
• 111
131 views

### Laymen's description of Laplace Smoothing

I have understood that Laplace Smoothing will provide a small non-zero value to the probability score. But still, I am missing something. Please if anyone can provide a better description, it will be ...
316 views

### Terminology for Bayesian Posterior Mean of Probability with Uniform Prior

If $p \sim$ Uniform$(0,1)$, and $X \sim$ Bin$(n, p)$, then the posterior mean of $p$ is given by $\frac{X+1}{n+2}$. Is there a common name for this estimator? I've found it solves lots of people's ...
• 21.4k
1k views

• 779
409 views

### Markov chain getting stuck due to insufficient data samples

There is a lot of theory on Markov models and output generation out there, but I cannot locate any information on models getting stuck. I'm trying to create a model of a data set using a Markov model....
• 33
339 views

### Smoothing a 2-by-2 contingency table

I am trying to implement a system for automatic document categorization, where each document of a corpus belongs to some class. I define the following contingency table for every class C and every ...
• 31
4k views

### What's a good approach to estimate the probability of word frequencies?

I have a document corpus and I want to estimate the probability of occurrence of a certain word $w$. Simply calculating the frequencies and use such a number as an estimation is not a good choice. Is ...
• 205