# Importance sampling in mixed, discrete/continuous variables

Consider the following model

$$X \sim |\mathcal{N}(X;0,1)|$$ $$Y|X \sim Q(Y;X)$$ where I define $Q(Y=-x|X=x)$ with probability mass $\int_{-\infty}^{-x}\mathcal{N}(x;0,1)dx$, $Q(Y=+x|X=x)$ with probability mass $1-\int_{x}^{\infty}\mathcal{N}(x;0,1)dx$, and the density of $Q(Y|X=x)$ equivalent to a truncated Normal distribution in $(-x,x)$.

Assume now that I an unknown $x_{unk}$ is sampled from $|\mathcal{N}(X;0,1)|$ and that $y$ is sampled from $Q(Y|X=x_{unk})$. I am given $y$ and would like to approximate the posterior distribution, $P(X|Y=y)$.

Using importance sampling, I would like to take $N$ sample values for $X$ then weight them according to the "probability" of $Y|X$ such that for sufficiently many samples, $\mathbb{E}[X|Y=y] \approx \frac{1}{\sum_{i}w_{i}} \sum_{i} x_{i} w_{i}$.

Were $Q(Y|X)$ entirely continuous, I would use $w_{i} = Q(Y=y|X=x_{i})$. Were it entirely discrete, I would use the probability mass function instead. As $Y|X$ is distributed according to a sort of hybrid, what should my weights be?

-

[edited]

Restating: The problem you'd like to solve is to infer the posterior over $X$, $P(X|Y)$, using particle filtering. You draw samples $X_i$ from $P(X)$ and then convert them to particles $Y_i \sim Q(Y|X_i)$, i.e., by sampling a $Y$ particle for each $X$ particle from the conditional distribution $Q(Y|X)$. You then need to apply Bayes' rule now to get the posterior of interest:

$P(X|Y) \propto Q(Y|X)P(X)$.

You sampled your initial particles from $P(X)$, so all you need to do is weight them by weights $w_i = Q(Y_i|X_i)$ and you're done! Is that it?

If so, then I think the problem is this: Q(Y|X) has delta masses as a function of Y. But NOT as a function of $X$! That is, there is a point mass at $\pm x$ in the conditional distribution $Q(Y|X_i)$ as a function of $Y$ for a fixed value $X_i$. However, that's not the weight you need here. You want the likelihood, which is $Q(Y_i|X)$ considered as a function of $X$. (Namely, you want to evaluate that function at $X=X_i$).

This latter thing (the likelihood) has no point masses. I assume the sampling procedure for $Y|X$ is something like "add some Gaussian noise and then theshold". You actually get a continuous likelihood function over $X$ for each value of $Y_i$. How you compute it will depend on $Y_i$:

1. If $Y_i=x$ (the upper boundary), then the likelihood involves normcdf: it's the probability that $X$ plus some noise ended up $>x$. So it's a sigmoidal function that goes from zero to 1 as it crosses $x$, with slope that depends on the variance of the noise added to $Y$, and it stays at $+1$ for all larger values of $X$ out to infinity. (Note that this doesn't integrate to 1; it doesn't have to since it's a likelihood!)
2. If $Y_i \in (-x,x)$, the likelihood involves the (standard) normpdf$(Y_i,X_i,\sigma^2)$

But in both cases, you have a continuous function over $X$. There's no value of $X$ for which a particular observation of $Y$ has infinitely higher probability than the other values of $X$ (except for trivial examples like $Y_i=-x$ and $X_i\rightarrow +\infty$).

-
I don't think I've explained myself well. I am sampling $(X)_{i=1}^{N}$ then trying to use $w_i = Q(Y;X_i)$ as a weighting function such that $\frac{1}{\sum_{i=1}^{N}w_{i}} \sum_{i=1}^{N}X_{i}w_{i} \approx E[X|Y]$ for sufficiently large N. Think of $X$ as the "maximum distance" a laser range finder can report, after which it just reports its max. I would like to know the probability distribution of the finder's maximum range given a single observation. –  duckworthd Jul 11 '11 at 1:43
I'll edit my answer... –  jpillow Jul 11 '11 at 2:56
I think I'm still being unclear. I'm not trying to use $|N(0,1)|$ as a proposal distribution for $Q(Y;X)$. I am given a realization $y$ of $Y$ drawn from $Q(Y;X)$ for unknown $X$ and would like to approximate the posterior $P(X|y)$ with particles. The support of $X$ is $\mathbb{R}$ so all samples from $|N(0,1)|$ that are greater than or equal to $|y|$ COULD potentially have generated $y$, it's just that some are more likely than others. My primary difficulty is that I don't know if $y$ was generated from one of the deltas or from a continuous portion of $Q(Y;X)$. –  duckworthd Jul 11 '11 at 5:01
Sorry--your original question is actually very clear; I think I was just too hasty reading it (i.e., thought I knew what you were asking and spouted off). I'll try one more time—I think I have a partial answer that results from a confusion about whether we're taking $Q$ as a function of $X$ or $Y$. –  jpillow Jul 11 '11 at 5:58