1
$\begingroup$

Suppose I have two independent random deviates $A$ and $B$ sampled from Gaussian (Normal) distributions with means $\mu_a$ and $\mu_b$ and standard deviations $\sigma_a$ and $\sigma_b$. I can't observe $A$ or $B$ directly, but see only their difference $C = A - B$.

Given that I observe $C=c$, what's $Pr\{A = a | C=c\}$ ?

Seems like a job for Bayes' rule, and its easy to write down

$$Pr\{A=a | c\} = {Pr\{C | A\} Pr\{A\} \over Pr\{C\}}$$

From the assumptions above $Pr\{A\}$ ~ Normal($\mu_a, \sigma_a^2$), and $$Pr\{C | A\}$$ $$= Pr\{C=A-B | A\}$$ $$= Pr\{B = A-C | A\}$$, which is also ~ Normal($\mu_b, \sigma_b^2$) (we've conditioned on $A$, so we just want the probability that $B$ equals some value)

...however this leads to a dense thicket of algebra I can't seem to climb out of. Any handy tricks or references I should examine? Based on simulations the solution is Gaussian, but it's some complex function of $\mu_a, \sigma_a, \mu_b, and \sigma_b$, which I can't seem to derive. Thanks!

Improve this question
$\endgroup$
  • $\begingroup$ Isn't $A$ independent of $C$ and therefore $P(A) = P(A|C)$? $\endgroup$ – Vivek Subramanian Feb 27 '18 at 0:01
  • 1
    $\begingroup$ 1. $\mathbb{P}(A=a|C=c)=0$ since $A$ is a continuous random variable, you need to phrase this in terms of densities. 2. Are the means and standard deviations of $A$ and $B$ known? If they are you just phrase the problem in terms the joint distribution of A and B, transform it to the joint distribution of $C$ and $A$ (this is just a linear transformation), then find the conditional of $A$ given $C$. All of this is doable from the properties of the multivariate normal distribution. $\endgroup$ – aleshing Feb 27 '18 at 0:35
  • $\begingroup$ @Vivek, since C = A - B, then given a value for C we have some information about what A must have been, so I don't think they're independent (and from simulations they do not seem to be) $\endgroup$ – user2225493 Feb 27 '18 at 1:55
  • $\begingroup$ @marmle, thanks I've been sloppily using Pr{A=a} to denote probability density, but perhaps p(a)_A would be more clear, apologies $\endgroup$ – user2225493 Feb 27 '18 at 2:51
2
$\begingroup$

Given the specified distributions for $A$ and $B$, you have the initial joint distribution:

$$\begin{bmatrix} A \\ B \end{bmatrix} \sim \text{N} \Bigg( \begin{bmatrix} \mu_A \\ \mu_B \end{bmatrix} , \begin{bmatrix} \sigma_A^2 & 0 \\ 0 & \sigma_B^2 \end{bmatrix} \Bigg).$$

Applying the appropriate linear transformation gives the joint distribution of interest:

$$\begin{bmatrix} A \\ C \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 1 & -1 \end{bmatrix} \begin{bmatrix} A \\ B \end{bmatrix} \sim \text{N} \Bigg( \begin{bmatrix} \mu_A \\ \mu_A - \mu_B \end{bmatrix} , \begin{bmatrix} \sigma_A^2 & \sigma_A^2 \\ \sigma_A^2 & \sigma_A^2 + \sigma_B^2 \end{bmatrix} \Bigg).$$

(As you can see, the random variables $A$ and $C$ are not independent.) Using the standard rules for the conditional distribution of a multivariate normal distribution, we have the conditional distribution $A|C \sim \text{N} (\mu_*(C), \sigma_*^2)$ where:

$$\begin{matrix} \mu_*(C) \equiv \mu_A + \frac{\sigma_A^2}{\sigma_A^2 + \sigma_B^2}(C - \mu_A+\mu_B) & & \sigma_*^2 \equiv \frac{\sigma_A^2 \sigma_B^2}{\sigma_A^2 + \sigma_B^2} \end{matrix}.$$

So as you can see, observing $C$ allows you an imperfect glimpse into $A$. If $\sigma_A \gg \sigma_B$ then you get a good predictor of $A$ and if $\sigma_A \ll \sigma_B$ then you get a poor predictor of $A$.

Improve this answer
$\endgroup$
  • $\begingroup$ Nice! I was stuck because I was using the transformation matrix $[1 -1]^T$ which gives just the marginal of $C$ rather than the one you used which gives the joint on $A$ and $C$. Makes it easy to find the conditional. $\endgroup$ – Vivek Subramanian Feb 27 '18 at 5:30
  • $\begingroup$ This matches my simulation results almost perfectly, thank you! $\endgroup$ – user2225493 Feb 27 '18 at 15:04
0
$\begingroup$

Given $A \sim \textsf{N}(\mu_a,\sigma_a^2)$ and $B \sim \textsf{N}(\mu_b,\sigma_b^2)$ where $A$ and $B$ are independent. Let $C = A - B$. The joint distribution of $(A,C)$ is bivariate normal: \begin{equation} \begin{bmatrix} A \\ C \end{bmatrix} \sim \textsf{N}\left(\begin{bmatrix}\mu_a \\ \mu_a - \mu_b\end{bmatrix}, \begin{bmatrix}\sigma_a^2 & \sigma_a^2 \\ \sigma_a^2 & \sigma_a^2 + \sigma_b^2 \end{bmatrix}\right) . \end{equation} Bayes rule says \begin{equation} p(A|C) = \frac{p(A,C)}{P(C)}. \end{equation} Therefore, the distribution for $A$ given $C$ is \begin{equation} A|C \sim \textsf{N}(m,s^2) , \end{equation} where \begin{equation} s^2 = \left(\frac{1}{\sigma_a^2} + \frac{1}{\sigma_b^2}\right)^{-1} \end{equation} and \begin{equation} m = s^2\left(\frac{\mu_a}{\sigma_a^2} + \frac{C + \mu_b}{\sigma_b^2}\right) . \end{equation}

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.