2
$\begingroup$

This is a simple question. I am dealing with a "clipped" normal distribution -- say, $N(0,0.5)$ clipped between $[-1,1]$. I would like to calculate the "probability" of a sample, but I know that in $(-1,1)$, the probability of a single event is 0. However, the probability of 1 is $1-\text{cdf}(1,0,0.5)$ and -1 is $\text{cdf}(-1,0,0.5)$. How does one typically reconcile these sorts of differences?

My intuition would be to use epsilon balls within $(-1,1)$, but it's not quite the RIGHT thing to do...

|cite|improve this question|||||
$\endgroup$
  • $\begingroup$ by in (-1,1) the probability of a single event is 0 do you mean the probability of getting exactly a given value is infinitesimally small or that it is impossible to get any value > -1 and < 1 (i.e. the probability density function is zero) ? From the answer below I suspect you mean the former... $\endgroup$ – Andre Holzner Jul 5 '11 at 21:32
  • $\begingroup$ I mean that for any continuous random variable, the probability mass function of any discrete point has value 0. $\endgroup$ – duckworthd Jul 10 '11 at 21:27
4
$\begingroup$

There are two different object you might be interested in:

  1. the truncated normal distribution, which is a Gaussian whose range is restricted to lie within $[a,b]$. This is a density, so it's continuous (no need for epsilon balls).
  2. a random variable $Z$ that results from drawing a Gaussian random variable $X$ and then "thresholding" it, setting $Z=a$ if $X<a$ and to $Z=b$ if $X>b$, to $Z=X$ if $a\leq X \leq b$.

I'm guessing it's the latter object you're interested in. As you've noted, the probability of getting a sample on the border is non-zero: $P(Z=a) = \Phi((a-\mu)/\sigma)$ and $P(Z=b)=\Phi((\mu-b)/\sigma)$, where $\Phi$ is the standard Gaussian cdf, $\mu$ is the mean, and $\sigma$ the standard deviation.

However, there is zero probability of getting any particular sample within the range $(a,b)$, but the probability density within this range is simply that of the original Gaussian. (The probability of getting a sample from somewhere within this range is simply $1-P(Z=a)-P(Z=b)$.

The resulting object is therefore defined by "point masses" at $a$ and $b$ and a continuous density between $a$ and $b$.

|cite|improve this answer|||||
$\endgroup$
  • $\begingroup$ 2) is the right model. That was as far as I got as well, but my application in particular (a particle filter) relies on the ability to weight each particle according to the "probability" of an observation. If observations are all discrete I can use the Prob. Mass Func. or if they're all continuous I can use the Prob. Dens. Func., but what do I do when they're mixed? $\endgroup$ – duckworthd Jul 9 '11 at 22:41
  • $\begingroup$ Interesting question! I haven't thought about this before, but one possibility: do you have access to the variable before thresholding? (i.e., before it is clipped to (-1,1)?). In that case, you could assign these points the probability that they would have had under the pdf (but the location of these particles will now be -1 or +1. Your particles would still represent the distribution in question accurately, because the weighted sum of particles at +1 and -1 would roughly correspond to the amount of probability mass at these locations. Not clear if this achieves what you need, however. $\endgroup$ – jpillow Jul 10 '11 at 1:03
  • $\begingroup$ Another approach would be maintain a mixed discrete/continuous representation of your distribution: use particles only to represent the portion captured by the density and point masses to represent the probability mass assigned to +/-1. Any particles that land at +/-1 get absorbed by the point masses, and you can presumably sample how much probability mass should be redistributed as particles during each MCMC step. You'd have to think about the details to make sure they're done correctly (e.g., detailed balance holds), but I don't see any reason this wouldn't work. $\endgroup$ – jpillow Jul 10 '11 at 4:07
  • $\begingroup$ I'm sorry, I don't quite understand. If I keep particles of state x_t ~ P(x_t|y_{1:t-1}) and would like to weight them by the "probability" of observation y_t|x_t, how can point masses help? Their pdf value would be infinite and the pmf values of any sample in the continuous portion would be zero. Also, I don't see the connection between MCMC's detailed balance and a Particle Filter's weighting. $\endgroup$ – duckworthd Jul 10 '11 at 21:25
  • $\begingroup$ Sorry, you're right—detailed balance isn't relevant to particle filtering (which is technically "sequential monte carlo", not MCMC). Let me try again: are the point masses in $P(x_t|y_{1:t-1}$ at the same location as in $P(y_t|x_t)$? If not, then the probability of having a particle (from the first density) that ends up at the location of a point mass in second is 0. If they are in the same location (which I suspect is the case), then I don't think it's a problem to weight particles in the "density" region by the pdf and those in the point masses by the pmf. Continuing... $\endgroup$ – jpillow Jul 10 '11 at 21:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.