# Posterior distribution after observing only difference in Gaussians

Suppose I have two independent random deviates $A$ and $B$ sampled from Gaussian (Normal) distributions with means $\mu_a$ and $\mu_b$ and standard deviations $\sigma_a$ and $\sigma_b$. I can't observe $A$ or $B$ directly, but see only their difference $C = A - B$.

Given that I observe $C=c$, what's $Pr\{A = a | C=c\}$ ?

Seems like a job for Bayes' rule, and its easy to write down

$$Pr\{A=a | c\} = {Pr\{C | A\} Pr\{A\} \over Pr\{C\}}$$

From the assumptions above $Pr\{A\}$ ~ Normal($\mu_a, \sigma_a^2$), and $$Pr\{C | A\}$$ $$= Pr\{C=A-B | A\}$$ $$= Pr\{B = A-C | A\}$$, which is also ~ Normal($\mu_b, \sigma_b^2$) (we've conditioned on $A$, so we just want the probability that $B$ equals some value)

...however this leads to a dense thicket of algebra I can't seem to climb out of. Any handy tricks or references I should examine? Based on simulations the solution is Gaussian, but it's some complex function of $\mu_a, \sigma_a, \mu_b, and \sigma_b$, which I can't seem to derive. Thanks!

• Isn't $A$ independent of $C$ and therefore $P(A) = P(A|C)$? – Vivek Subramanian Feb 27 '18 at 0:01
• 1. $\mathbb{P}(A=a|C=c)=0$ since $A$ is a continuous random variable, you need to phrase this in terms of densities. 2. Are the means and standard deviations of $A$ and $B$ known? If they are you just phrase the problem in terms the joint distribution of A and B, transform it to the joint distribution of $C$ and $A$ (this is just a linear transformation), then find the conditional of $A$ given $C$. All of this is doable from the properties of the multivariate normal distribution. – aleshing Feb 27 '18 at 0:35
• @Vivek, since C = A - B, then given a value for C we have some information about what A must have been, so I don't think they're independent (and from simulations they do not seem to be) – user2225493 Feb 27 '18 at 1:55
• @marmle, thanks I've been sloppily using Pr{A=a} to denote probability density, but perhaps p(a)_A would be more clear, apologies – user2225493 Feb 27 '18 at 2:51

Given the specified distributions for $A$ and $B$, you have the initial joint distribution:

$$\begin{bmatrix} A \\ B \end{bmatrix} \sim \text{N} \Bigg( \begin{bmatrix} \mu_A \\ \mu_B \end{bmatrix} , \begin{bmatrix} \sigma_A^2 & 0 \\ 0 & \sigma_B^2 \end{bmatrix} \Bigg).$$

Applying the appropriate linear transformation gives the joint distribution of interest:

$$\begin{bmatrix} A \\ C \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 1 & -1 \end{bmatrix} \begin{bmatrix} A \\ B \end{bmatrix} \sim \text{N} \Bigg( \begin{bmatrix} \mu_A \\ \mu_A - \mu_B \end{bmatrix} , \begin{bmatrix} \sigma_A^2 & \sigma_A^2 \\ \sigma_A^2 & \sigma_A^2 + \sigma_B^2 \end{bmatrix} \Bigg).$$

(As you can see, the random variables $A$ and $C$ are not independent.) Using the standard rules for the conditional distribution of a multivariate normal distribution, we have the conditional distribution $A|C \sim \text{N} (\mu_*(C), \sigma_*^2)$ where:

$$\begin{matrix} \mu_*(C) \equiv \mu_A + \frac{\sigma_A^2}{\sigma_A^2 + \sigma_B^2}(C - \mu_A+\mu_B) & & \sigma_*^2 \equiv \frac{\sigma_A^2 \sigma_B^2}{\sigma_A^2 + \sigma_B^2} \end{matrix}.$$

So as you can see, observing $C$ allows you an imperfect glimpse into $A$. If $\sigma_A \gg \sigma_B$ then you get a good predictor of $A$ and if $\sigma_A \ll \sigma_B$ then you get a poor predictor of $A$.

• Nice! I was stuck because I was using the transformation matrix $[1 -1]^T$ which gives just the marginal of $C$ rather than the one you used which gives the joint on $A$ and $C$. Makes it easy to find the conditional. – Vivek Subramanian Feb 27 '18 at 5:30
• This matches my simulation results almost perfectly, thank you! – user2225493 Feb 27 '18 at 15:04

Given $A \sim \textsf{N}(\mu_a,\sigma_a^2)$ and $B \sim \textsf{N}(\mu_b,\sigma_b^2)$ where $A$ and $B$ are independent. Let $C = A - B$. The joint distribution of $(A,C)$ is bivariate normal: \begin{equation} \begin{bmatrix} A \\ C \end{bmatrix} \sim \textsf{N}\left(\begin{bmatrix}\mu_a \\ \mu_a - \mu_b\end{bmatrix}, \begin{bmatrix}\sigma_a^2 & \sigma_a^2 \\ \sigma_a^2 & \sigma_a^2 + \sigma_b^2 \end{bmatrix}\right) . \end{equation} Bayes rule says \begin{equation} p(A|C) = \frac{p(A,C)}{P(C)}. \end{equation} Therefore, the distribution for $A$ given $C$ is \begin{equation} A|C \sim \textsf{N}(m,s^2) , \end{equation} where \begin{equation} s^2 = \left(\frac{1}{\sigma_a^2} + \frac{1}{\sigma_b^2}\right)^{-1} \end{equation} and \begin{equation} m = s^2\left(\frac{\mu_a}{\sigma_a^2} + \frac{C + \mu_b}{\sigma_b^2}\right) . \end{equation}