I have a very simple graphical model to describe the relationship between two categorical variables $c \in \{0,1\}$ and $l \in \{A,B,C\}$:

$$c \rightarrow l$$

I know all the conditional probabilities $p(l \mid c)$ through a lookup table. Calculating the posterior $p(c \mid l)$ is simple using Bayes' rule.

However, what if I observe the labels $l$ with some uncertainty? That means my observation would be a tuple $(p_A, p_B, p_C)$, i.e. $(0.8, 0.1, 0.1)$ instead of observing $l=A$.

Is there a way to calculate the posterior probability accordingly? I assume I can find some sort of weighted contribution (as below), but I couldn't figure out how to formulate this.

$$ p(c \mid \mbox{observed tuple} ) = p_A ~ p(c \mid l=A) + p_B ~ p(c | l=B) + p_C ~ p(c \mid l=C) $$

Thanks for any help!

  • 1
    $\begingroup$ It will be helpful for deriving a posterior to put a node in your network, say $l^*$ representing the true value which is obscured by measurement error $l$. Also your tuple is an assertion about the structure of the measurement error rather than an observation. $\endgroup$ Apr 1, 2016 at 17:10
  • $\begingroup$ Thanks for your comment. So that would mean $p(l^*=A \mid l) = p_A$ if I understand correctly - makes sense! But where in the network would you insert the new node? $\endgroup$
    – Lisa
    Apr 1, 2016 at 17:18
  • $\begingroup$ Can you please elaborate on your second sentence? I'm not sure I understand what you mean by "structure of the measurement error". $\endgroup$
    – Lisa
    Apr 1, 2016 at 17:21
  • $\begingroup$ The idea would be that the graph has $c \rightarrow l^*$. Then if $l^*$ has classical measurement error then there would be $l^* \rightarrow l$ and if it has Berkson measurement error there would be a $l \rightarrow l^*$. $\endgroup$ Apr 1, 2016 at 21:33
  • 1
    $\begingroup$ How about: $p(c \mid l) = \sum_{l^*} p(c \mid l^*, l)\, p(l^* \mid l) = \sum_{l^*} p(c \mid l^*)\, p(l^* \mid l)$ where the first step is the law of total probability and the second follows from a conditional independence relation implied by your graph. The second term is your tuple and the first term I think you implied you had already, or could construct from $p(l^* \mid c)$ and $p(c)$. $\endgroup$ Apr 4, 2016 at 18:32

1 Answer 1


You want to know $$ p(c \mid l) $$ where $c$ is observed, but $l$ is a noisy realization of an unobserved $l^*$ which is generated by $c$. If you could observe $l^*$ you would and you'd condition on it, but you don't so you can't.

A simple graph consistent with this measurement error problem is $$ c \rightarrow l^* \rightarrow l $$ You assume that $l^*$ has the same possible values as $l$, so I'll assume that too, though it's not essential. And you even know $p(l^* \mid l)$, at least for the value of $l$ you care about; that's your tuple. (Whether you got that directly or had to infer it with extra distributional assumptions is determined by whether you have a classical or Berkson measurement error setup, but we'll ignore all that). Moreover you also assume you know how to get a distribution of $c$ given $l^*$, perhaps because you know $p(c)$ and $p(l^* \mid c)$ and can apply Bayes theorem.

With all this in hand, the conditional probability you want is $$ p(c \mid l) = \sum_{l^*} p(c \mid l^*, l)\, p(l^* \mid l) = \sum_{l^*} p(c \mid l^*)\, p(l^* \mid l) $$ where the first step is the rule of total probability and the second step comes from the structure of the graph: $c$ is independent of $l$ conditional on $l^*$.

The result is, I think, pretty intuitive: we should average over all the $l^*$ involving possible routes between $c$ and $l$, weighting by what we know about their relationships to $l^*$.

If there are more variables involved then the result might be different and perhaps much messier. In particular, the conditional independence need not hold. Nevertheless, writing down the graph and cranking through the rules of probability will get you where you need to go.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.