Posterior distribution after observing only difference in Gaussians

Question

Suppose I have two independent random deviates $A$ and $B$ sampled from Gaussian (Normal) distributions with means $\mu_a$ and $\mu_b$ and standard deviations $\sigma_a$ and $\sigma_b$. I can't observe $A$ or $B$ directly, but see only their difference $C = A - B$.

Given that I observe $C=c$, what's $Pr\{A = a | C=c\}$ ?

Seems like a job for Bayes' rule, and its easy to write down

$$Pr\{A=a | c\} = {Pr\{C | A\} Pr\{A\} \over Pr\{C\}}$$

From the assumptions above $Pr\{A\}$ ~ Normal($\mu_a, \sigma_a^2$), and $$Pr\{C | A\}$$ $$= Pr\{C=A-B | A\}$$ $$= Pr\{B = A-C | A\}$$, which is also ~ Normal($\mu_b, \sigma_b^2$) (we've conditioned on $A$, so we just want the probability that $B$ equals some value)

...however this leads to a dense thicket of algebra I can't seem to climb out of. Any handy tricks or references I should examine? Based on simulations the solution is Gaussian, but it's some complex function of $\mu_a, \sigma_a, \mu_b, and \sigma_b$, which I can't seem to derive. Thanks!

Answer 1

Given the specified distributions for $A$ and $B$, you have the initial joint distribution:

$$\begin{bmatrix} A \\ B \end{bmatrix} \sim \text{N} \Bigg( \begin{bmatrix} \mu_A \\ \mu_B \end{bmatrix} , \begin{bmatrix} \sigma_A^2 & 0 \\ 0 & \sigma_B^2 \end{bmatrix} \Bigg).$$

Applying the appropriate linear transformation gives the joint distribution of interest:

$$\begin{bmatrix} A \\ C \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 1 & -1 \end{bmatrix} \begin{bmatrix} A \\ B \end{bmatrix} \sim \text{N} \Bigg( \begin{bmatrix} \mu_A \\ \mu_A - \mu_B \end{bmatrix} , \begin{bmatrix} \sigma_A^2 & \sigma_A^2 \\ \sigma_A^2 & \sigma_A^2 + \sigma_B^2 \end{bmatrix} \Bigg).$$

(As you can see, the random variables $A$ and $C$ are not independent.) Using the standard rules for the conditional distribution of a multivariate normal distribution, we have the conditional distribution $A|C \sim \text{N} (\mu_*(C), \sigma_*^2)$ where:

$$\begin{matrix} \mu_*(C) \equiv \mu_A + \frac{\sigma_A^2}{\sigma_A^2 + \sigma_B^2}(C - \mu_A+\mu_B) & & \sigma_*^2 \equiv \frac{\sigma_A^2 \sigma_B^2}{\sigma_A^2 + \sigma_B^2} \end{matrix}.$$

So as you can see, observing $C$ allows you an imperfect glimpse into $A$. If $\sigma_A \gg \sigma_B$ then you get a good predictor of $A$ and if $\sigma_A \ll \sigma_B$ then you get a poor predictor of $A$.

Answer 2

Given $A \sim \textsf{N}(\mu_a,\sigma_a^2)$ and $B \sim \textsf{N}(\mu_b,\sigma_b^2)$ where $A$ and $B$ are independent. Let $C = A - B$. The joint distribution of $(A,C)$ is bivariate normal: \begin{equation} \begin{bmatrix} A \\ C \end{bmatrix} \sim \textsf{N}\left(\begin{bmatrix}\mu_a \\ \mu_a - \mu_b\end{bmatrix}, \begin{bmatrix}\sigma_a^2 & \sigma_a^2 \\ \sigma_a^2 & \sigma_a^2 + \sigma_b^2 \end{bmatrix}\right) . \end{equation} Bayes rule says \begin{equation} p(A|C) = \frac{p(A,C)}{P(C)}. \end{equation} Therefore, the distribution for $A$ given $C$ is \begin{equation} A|C \sim \textsf{N}(m,s^2) , \end{equation} where \begin{equation} s^2 = \left(\frac{1}{\sigma_a^2} + \frac{1}{\sigma_b^2}\right)^{-1} \end{equation} and \begin{equation} m = s^2\left(\frac{\mu_a}{\sigma_a^2} + \frac{C + \mu_b}{\sigma_b^2}\right) . \end{equation}