Two Distributions, One a Sum: Discerning likelihood given error

Question

Given $X, Y$ independent and non-normal, I'm recording histograms of $X$ and of $Z = X + Y$, sampled when $Y$ is not present and when it is, respectfully. I'm trying to figure out $Var(Y)$ and its sampling error.

I thought I could just calculate $Var(Y) = Var(Z) - Var(X)$ and use the sum of the sampling variances to produce the variance related to the final error, but the process breaks down when $Y$ is small or might not exist, occasionally resulting in a negative difference that has no meaning to me as a variance. That is, $Var(Z) - Var(X) < 0$ possibly due to sampling error.

When this difference is negative, it seems either the proposition that $Z = X + Y$ is false, or simply that the real value is close to zero and not enough samples have been taken to get the measured result close enough to zero yet.

Is there a way to determine the likelihood that the proposition is false or true when the measured value is very negative?

Answer 1

Sample variances operate differently to the underlying variance parameters. If you would like to express the sample variance of $Y$ in terms of the sample moments of $X$ and $Z$ then you will need to formulate an appropriate decomposition formula. Assuming you are dealing with $n$ paired values (it only makes sense to add them if they are paired) you have $\bar{Y} = \bar{Z} - \bar{X}$, which then gives:

$$\begin{equation} \begin{aligned} S_Y^2 &= \frac{1}{n - 1} \sum_{i=1}^n (Y_i - \bar{Y})^2 \\[6pt] &= \frac{1}{n - 1} \sum_{i=1}^n (Z_i - X_i - \bar{Z} + \bar{X})^2 \\[6pt] &= \frac{1}{n - 1} \sum_{i=1}^n ((Z_i - \bar{Z}) - (X_i - \bar{X}))^2 \\[6pt] &= \frac{1}{n - 1} \sum_{i=1}^n \big[ (Z_i - \bar{Z})^2 - 2(Z_i - \bar{Z})(X_i - \bar{X}) + (X_i - \bar{X})^2 \big] \\[6pt] &= S_Z^2 -2 \cdot S_{X,Z} + S_X^2, \\[6pt] \end{aligned} \end{equation}$$

where we have used the sample covariance:

$$S_{X,Z} = \frac{1}{n - 1} \sum_{i=1}^n (Z_i - \bar{Z})(X_i - \bar{X}).$$

Since $Z_i = X_i + Y_i$ it is worth noting that $S_{X,Z} = S_{X,Y} + S_X^2$. Since $X$ and $Y$ are independent, as $n \rightarrow \infty$ we have $S_{X,Z} \rightarrow S_X^2$ so that $S_Y^2 \rightarrow S_Z^2 - S_X^2$.

Answer 2

I'm tentatively considering this answer:

The likelihood I want is the likelihood that the $X$ I recorded has something in it that is not present in the $Z$ I recorded.

So if $Var(X) > Var(Z)$ I consider where in the sampling variance chi-squared distribution of Z I find $Var(X)$, and if this is beyond a threshold, I report the data as faulty.

I found a description of the sampling variance for non-normal distributions at https://stats.stackexchange.com/q/347476 . I used Wikipedia to find formulas for the variance and CDF of the chi-squared distribution: https://en.wikipedia.org/wiki/Chi-squared_distribution . I use the CDF to calculate my likelihood stat.

I would love to know whether this is correct, though!

Answer 3

Possibly relevant simulation.

I model $S$ as the sum of ten observations from $\mathsf{Norm}(\mu=10, \sigma=5),$ corresponding to your $Z = X+Y,$ and $T$ as the sum of ten observations from $\mathsf{Norm}(9, 3),$ corresponding to your (separately observed) $X$. Then $D$ corresponds to your attempt to recover the distribution of $Y.$ However $D$ is approximately $\mathsf{Norm}(100, 50).$

This formulation models that $X$'s that are components of $S$ as independent of the $X$'s that are simulated separately as $T$'s. The $D$'s have the same mean as the $Y$'s, but not the same variance.

set.seed(113);  m = 10^6; n = 100
d = replicate(m,  sum(rnorm(n, 10, 4)) - sum(rnorm(n, 9, 3)))
mean(d);  sd(d);  mean(d < 0)
[1] 99.99261   # aprx E(D)
[1] 49.99622   # aprx SD(D)
[1] 0.022642   # aprx P(D < 0)

Notice that more than 2% of the simulated $D$'s are negative.

hist(d, prob=T, br = 25, col="skyblue2")
curve(dnorm(x, 100, 50), add=T, lwd=2)
abline(v = 0, col="green2")