1
$\begingroup$

I need to calculate a threshold value to get rid of Poisson distributed noise in an image to perform a cluster analysis on the image.

The image is the representation of a signal, whose datapoints were binned into container (= single pixel of the image). For every datapoint in a container, its level is increased by one. The threshold t is used to remove pixel with intensity level < threshold t.

Poisson distribution is an assumption based on previous work on the signal.

For that I have the following equation:

$$ \sum\limits_{k=0}^t \left[P(k=t) = \frac{\lambda^ke^{-\lambda}}{k!}\right] > 0.999, $$

$\lambda$ is known. I only need to find $t$.

My naive approach was to calculate the sum for t=1, t=2,... until sum > 0.999. However this gives me OverflowErrors in Python.

Am I on the right path or, if not, how can I find t?

$\endgroup$
  • $\begingroup$ Could you give us more details? What kind of noise? For what data? Why do you need to remove it? How do you know it is Poisson distributed? $\endgroup$ – Tim Dec 8 '18 at 20:04
0
$\begingroup$

I guess, your summand is just $P(k)$, not $P(k=t)$. Assuming RHS of equal sign is correct, you normally need to use the CDF of Poisson Distribution, which has a non-trivial closed form involving Incomplete Gamma Function. So, practically, what you're doing is correct. But, you should be careful when calculating factorials and exponentials involving large numbers. You can convert your calculations as: $$\frac{e^{-\lambda}\lambda^k}{k!}=e^{-\lambda+k\log\lambda-\sum_{m=1}^k\log m}$$ And first calculate the exponent part, i.e. $-\lambda+k\log\lambda-\sum_{m=1}^k\log m$, then exponentiate for numerical stability. Therefore, you won't need to divide a very large exponential by a very large factorial number.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.