According to sums of Independent Normal Random Variables enter image description here

Let's say we have two samples. Sample 1 follows a N(u1, var1) and Sample 2 follows a N(u2, var2),

Case 1: I take one subject from each sample, those two subjects will follow a N(u1+u2,var1+var2), easy to be proved by using moment-generating function of a linear combination of independent random variables.

Case 2: I have a different situation now, let's say I mixed two samples first, then I take one subject in the mixed sample, it's intuitive to consider it still follow a N(u1+u2,var1+var2). how can I prove it?

  • 1
    $\begingroup$ You can't because it is not true. The distribution in that case is more like a mixture of normals. $\endgroup$ – JohnK Jan 23 '16 at 0:54
  • $\begingroup$ @JohnK what's the correct normal model ( u=?, var=?)in the second case?If it is not the case, how can I prove that in ANOVA,SSTotal follows a chi square distribution with n-1 df, which is the same as case 2 $\endgroup$ – whoisit Jan 23 '16 at 0:59

There is a difference between summing random variables and mixing them. In your post your mixing random variables which leads to a normal mixture distribution.

To see the difference, let $$ X_1,X_2,...,X_n \stackrel{iid}{\sim} N(\mu_1,\sigma_1^2) $$ and $$ Y_1,Y_2,...,Y_n \stackrel{iid}{\sim} N(\mu_2,\sigma_2^2) $$ where $X_i$ and $Y_i$ are independent for all $i \in \{1,...,n\}$.

If you created a new variable $Z_1,Z_2,...,Z_n$ where $Z_i = X_i+Y_i$, then $Z_i \stackrel{iid}{\sim} N(\mu_1+\mu_2,\sigma_1^2+\sigma_2^2)$. However, this is not what you said in your post. What you said was that you "mixed two samples".

Suppose that you did mix these samples and the observations where unlabeled (meaning that you don't know which original sample each observation came from). Denote the mixed sample as $A_1, A_2,...,A_{2n}$. For each $i$ there is a 50% chance that $A_i$ came from sample 1 and a 50% chance it came from sample 2. Thus $$ A_i \stackrel{iid}{\sim} .5 \times \phi(A_i|\mu_1,\sigma_1^2) +.5 \times \phi(A_i|\mu_2,\sigma_2^2) $$ where $\phi$ is the normal probability density function. The above distribution is called a mixture and it differs greatly from the normal distribution you proposed in your post example. The below plot is an example of a mixture between two normals a $N(-3,1)$ and $N(,2,0.5^2)$ compared to the distirubtion of their sum.

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.