# Bayesian inference when observed variable contains uncertainty

I have a very simple graphical model to describe the relationship between two categorical variables $c \in \{0,1\}$ and $l \in \{A,B,C\}$:

$$c \rightarrow l$$

I know all the conditional probabilities $p(l \mid c)$ through a lookup table. Calculating the posterior $p(c \mid l)$ is simple using Bayes' rule.

However, what if I observe the labels $l$ with some uncertainty? That means my observation would be a tuple $(p_A, p_B, p_C)$, i.e. $(0.8, 0.1, 0.1)$ instead of observing $l=A$.

Is there a way to calculate the posterior probability accordingly? I assume I can find some sort of weighted contribution (as below), but I couldn't figure out how to formulate this.

$$p(c \mid \mbox{observed tuple} ) = p_A ~ p(c \mid l=A) + p_B ~ p(c | l=B) + p_C ~ p(c \mid l=C)$$

Thanks for any help!

• It will be helpful for deriving a posterior to put a node in your network, say $l^*$ representing the true value which is obscured by measurement error $l$. Also your tuple is an assertion about the structure of the measurement error rather than an observation. Apr 1, 2016 at 17:10
• Thanks for your comment. So that would mean $p(l^*=A \mid l) = p_A$ if I understand correctly - makes sense! But where in the network would you insert the new node?
– Lisa
Apr 1, 2016 at 17:18
• Can you please elaborate on your second sentence? I'm not sure I understand what you mean by "structure of the measurement error".
– Lisa
Apr 1, 2016 at 17:21
• The idea would be that the graph has $c \rightarrow l^*$. Then if $l^*$ has classical measurement error then there would be $l^* \rightarrow l$ and if it has Berkson measurement error there would be a $l \rightarrow l^*$. Apr 1, 2016 at 21:33
• How about: $p(c \mid l) = \sum_{l^*} p(c \mid l^*, l)\, p(l^* \mid l) = \sum_{l^*} p(c \mid l^*)\, p(l^* \mid l)$ where the first step is the law of total probability and the second follows from a conditional independence relation implied by your graph. The second term is your tuple and the first term I think you implied you had already, or could construct from $p(l^* \mid c)$ and $p(c)$. Apr 4, 2016 at 18:32

You want to know $$p(c \mid l)$$ where $c$ is observed, but $l$ is a noisy realization of an unobserved $l^*$ which is generated by $c$. If you could observe $l^*$ you would and you'd condition on it, but you don't so you can't.
A simple graph consistent with this measurement error problem is $$c \rightarrow l^* \rightarrow l$$ You assume that $l^*$ has the same possible values as $l$, so I'll assume that too, though it's not essential. And you even know $p(l^* \mid l)$, at least for the value of $l$ you care about; that's your tuple. (Whether you got that directly or had to infer it with extra distributional assumptions is determined by whether you have a classical or Berkson measurement error setup, but we'll ignore all that). Moreover you also assume you know how to get a distribution of $c$ given $l^*$, perhaps because you know $p(c)$ and $p(l^* \mid c)$ and can apply Bayes theorem.
With all this in hand, the conditional probability you want is $$p(c \mid l) = \sum_{l^*} p(c \mid l^*, l)\, p(l^* \mid l) = \sum_{l^*} p(c \mid l^*)\, p(l^* \mid l)$$ where the first step is the rule of total probability and the second step comes from the structure of the graph: $c$ is independent of $l$ conditional on $l^*$.
The result is, I think, pretty intuitive: we should average over all the $l^*$ involving possible routes between $c$ and $l$, weighting by what we know about their relationships to $l^*$.