Mixing probabilities and probability densities

Question

I'm currently working on a Bayesian network designed to find the probabilities for various lung diseases. In the network there are, among others, a normally distributed random variable (body temperature) causally dependent on a binary one.

In order to reach a diagnosis MCMC sampling is used (Metropolis in Gibbs). When generating a sample, random variables are assigned new values to generate a candidate for a sample and the probabilities are compared with the unaltered variant. ($P_{new}/P_{old}$). The new values are accepted with probability $min(1, (P_{new}/P_{old}))$

When this comparison is made I multiply the probabilities of each value (given the values of it's dependencies) to get the probability of the current set of variables. However, I'm also multiplying these probabilities with a probability density value from the normally distributed random variable (from dnorm(x, mean, std) in R, where x in this case is an observed temperature). I figured this would be OK since I'm doing the same for both $P_{new}$ and $P_{old}$, but my results leads me to suspect otherwise. Is this an okay way to compare the proposed values to the old ones? If not, what am I getting wrong and what should I do to get it right?

EDIT: Here comes an attempt to make the question clearer as per Xi'an's request.

I'm comparing $p_{old} = P(X=x_{old}, Y=y)$ and $p_{new} = P(X=x_{new}, Y=y)$ where$X$ is categorically distributed and is $Y\thicksim N(\mu , \sigma)$ and casually dependent on $X$. I want to see if $(p_{new}/p_{old})> 1$.

To do this I calculate: $$ P(X=x_{old}, Y=y) = P(X=x_{old})\cdot P(Y=y |X=x_{old}) $$ and $$ P(X=x_{new}, Y=y) = P(X=x_{new})\cdot P(Y=y |X=x_{new}) $$ I can easily look up the value of $P(X=x_{old})$ and $P(X=x_{new})$ in my trained network. However, because $Y$ is normally distributed I've used R's dnorm(y, mean, std) R, where y is the $y$ (body temperature) in $P(Y=y |X=x_{new})$ and wheremean and std are the appropriately trained values from my historical data. I know that dnorm(y, mean, std) returns a probability density value $f(y)$, so what I'm actually calculating is

$$ P(X=x_{old}, Y=y) \propto P(X=x_{old})\cdot f_{x_{old}}(y) $$ and $$ P(X=x_{new}, Y=y) \propto P(X=x_{new})\cdot f_{x_{new}}(y) $$ I added $\propto$ in the equations to symbolize how I've though about it. Since I'm doing the same for both $p_{old}$ and $p_{new}$ when checking if $(p_{new}/p_{old})> 1$ I have been thinking this was OK. Now, I'm doubting if that's the case. If not, what should I do instead?

Answer 1

The setting sounds to be one where the quantity to be simulated, $X$, is discrete (e.g., integer), while an observable $Y$ indexed by $X$ is continuous. The posterior distribution of $X$ given $Y=y$ has probability mass function $$\Bbb P(X=k) f_k(y) \big/ \sum_m P(X=m) f_m(y)$$ A Metropolis-Hastings scheme aiming at this posterior will make a proposal $x_\text{new}$ from a symmetric distribution (around $x_\text{old}$) and accept it with probability $$\min\{1,\Bbb P(X=x_\text{new}|Y=y)\big/\Bbb P(X=x_\text{old}|Y=y)\}$$ equal to $$\min\{1,\Bbb P(X=x_\text{new})f_{x_\text{new}}(y)\big/\Bbb P(X=x_\text{old})f_{x_\text{old}}(y)\}$$

Answer 2

Proportionality is not fixing it, the point is different. Even if density and mass probability are two different things, and it's important to understand it when first approaching statistics, from a mathematical point of view they are actually very similar, so they are often expressed with the same symbol $p(\cdot)$. For instance, conditional and joint density and mass probability work the same way.

I can see why $p(X, Y)$ confuses you here, it is not continuous, nor discrete. Actually it is continuous on one dimension and discrete on the other. If you marginalize it with regards to $X$ you get a probability density, if you marginalize it from the other side you get a mass probability.

Time has come, my friend, that you start getting familiar with general probability function $p(\cdot)$. And it's actually very easy: just treat everything as it is simple probability, from a mathematical approach, they are practically the same thing.

How is it possible to put together two things conceptually very distinct? The difference only lays in the domain of the random variables: if I integrate $Y$ over its domain, I get 1; if I integrate $X$ over its domain, I still get 1, because its domain is discrete, and integrals measure it giving unit value to every points it counts. So integrals over discrete domains become sums. And that is the only difference.

Don't be confused though: in general $P(\cdot) \not = p(\cdot)$. That's only true for discrete distribution because $P$ maps measures (integrals) of the domain of random variables, and, as I said, in discrete domains every point has value 1, so $P(x) = 1\cdot p(x) = p(x)$.