Questions tagged [underflow]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers

Multivariate Normal Underflow

Good day everyone At the moment I am attempting to write code in R to calculate the following. $$ \tau_{k j}^{(m)}=\frac{\pi_{k}^{(m)} f_{k}\left(x_{j} ; \theta_{k}^{(m)}\right)}{f\left(x_{j} ; \Theta^...
Susan-l3p's user avatar
2 votes
0 answers

Avoiding underflow with arbitrarily large number of data points in a joint likelihood calculation? [closed]

I am trying to compute a Bayesian posterior distribution, given a large number of data points. I’ve found that as the number of data points increases, the joint likelihood of the data underflows to ...
Ravi P.'s user avatar
  • 21
1 vote
0 answers

Insufficient floating point precision for the correct computation of a density

I'm using the Metorpolis-Hastings algorithm in a setting where the acceptance function is essentially of the form $$\alpha(x,y)=1\wedge\frac{u(x,y)}{v(x,y)},$$ where $$u(x,y)=p+(1-p)\prod_{i=1}^mu_i(x,...
0xbadf00d's user avatar
  • 141
0 votes
0 answers

Equivalent of log sum exp trick for subtraction [duplicate]

I have two small positive real numbers $u, w$ such that $u > w$. Given $\log(u), \log(w)$ I'd like to find a numerically stable way to calculate $\log(u - w)$. One possible way of transforming the ...
marcusy's user avatar
  • 43
2 votes
1 answer

Computation within log space

What is the conversion of the following equation into log space? $bf2 = 1 + (p * (bf1 - 1))$ Given log.bf1 (log Bayes factor), how do I get to log.bf2 without having to compute bf1, but instead ...
user3302113's user avatar
7 votes
2 answers

Subtracting very small probabilities - How to compute? [duplicate]

This question is an extension of a related question about adding small probabilities. Suppose you have log-probabilities $\ell_1 \geqslant \ell_2$, where the corresponding probabilities $\exp(\ell_1)$...
Ben's user avatar
  • 119k
3 votes
1 answer

Vectorised computation of logsumexp

In this related post there is an explanation of how you can add together two very small probabilities using the logsumexp function, and how this can be programmed into base ...
Ben's user avatar
  • 119k
4 votes
1 answer

Adding very small probabilities—How to compute?

In some problems, probabilities are so small that they are best represented in computational facilities as log-probabilities. Computational problems can arise when you try to add these small ...
Ben's user avatar
  • 119k
1 vote
0 answers

Underflow when estimating marginal likelihood via bridge sampling

I try to use an iterative procedure to estimate the marginal likelihood in a Bayesian setting for model selection. In case you are interested in the specifics of bridge sampling in my application, see ...
yrx1702's user avatar
  • 690
-1 votes
1 answer

Avoiding Matlab's underflow prevention in backprop, due to performance cost

In Matlab, I understand that if a number gets closer to zero than realmin, then Matlab converts the double to a denorm . I am noticing this causes significant ...
rnoodle's user avatar
  • 223
2 votes
1 answer

How to randomly sample values given the extremly small and large log-probabilities? [duplicate]

Assume a long list of log-values. The list consists of very small negative numbers, very large negative numbers, as well as very large positive numbers. To avoid numerical overflow/underflow, I need ...
user3639557's user avatar
  • 1,432
9 votes
1 answer

Kernel density estimation on bounded support?

I was looking for some way to deal with boundary bias of kde in case of a unit interval. One example is an usage of Chen estimators (or Beta estimators; an example might be seen here: http://stats-www....
Tzab's user avatar
  • 91
1 vote
0 answers

Sum of very low probability [duplicate]

I have a score of some feature, $F_1$ and $F_2$ where this score is the logarithm of probability. This score is very low and very sparse, for example i have: $F_1$ with score $-800$ $F_2$ with ...
Neptune's user avatar
  • 599
17 votes
3 answers

Example of how the log-sum-exp trick works in Naive Bayes

I have read about the log-sum-exp trick in many places (e.g. here, and here) but have never seen an example of how it is applied specifically to the Naive Bayes classifier (e.g. with discrete features ...
Josh's user avatar
  • 4,058