Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
If we have 2 normal, uncorrelated random variables $X_1, X_2$ then we can create 2 correlated random variables with the formula
$Y=\rho X_1+ \sqrt{1-\rho^2} X_2$
and then $Y$ will have a correlation $\rho$ with $X_1$.
Can someone explain where this formula comes from?
Suppose you want to find a linear combination of $X_1$ and $X_2$ such that
$$ \text{corr}(\alpha X_1 + \beta X_2, X_1) = \rho $$
Notice that if you multiply both $\alpha$ and $\beta$ by the same (non-zero) constant, the correlation will not change. Thus, we're going to add a condition to preserve variance: $\text{var}(\alpha X_1 + \beta X_2) = \text{var}(X_1)$
This is equivalent to
$$ \rho = \frac{\text{cov}(\alpha X_1 + \beta X_2, X_1)}{\sqrt{\text{var}(\alpha X_1 + \beta X_2) \text{var}(X_1)}} = \frac{\alpha \overbrace{\text{cov}(X_1, X_1)}^{=\text{var}(X_1)} + \overbrace{\beta \text{cov}(X_2, X_1)}^{=0}}{\sqrt{\text{var}(\alpha X_1 + \beta X_2) \text{var}(X_1)}} = \alpha \sqrt{\frac{\text{var}(X_1)}{\alpha^2 \text{var}(X_1) + \beta^2 \text{var}(X_2)}} $$
Assuming both random variables have the same variance (this is a crucial assumption!) ($\text{var}(X_1) = \text{var}(X_2)$), we get
$$ \rho \sqrt{\alpha^2 + \beta^2} = \alpha $$
There are many solutions to this equation, so it's time to recall variance-preserving condition:
$$ \text{var}(X_1) = \text{var}(\alpha X_1 + \beta X_2) = \alpha^2 \text{var}(X_1) + \beta^2 \text{var}(X_2) \Rightarrow \alpha^2 + \beta^2 = 1 $$
And this leads us to
$$ \alpha = \rho \\ \beta = \pm \sqrt{1-\rho^2} $$
UPD. Regarding the second question: yes, this is known as whitening.
The equation is a simplified bivariate form of Cholesky decomposition. This simplified equation is sometimes called the Kaiser-Dickman algorithm (Kaiser & Dickman, 1962).
Note that $X_1$ and $X_2$ must have the same variance for this algorithm to work properly. Also, the algorithm is typically used with normal variables. If $X_1$ or $X_2$ are not normal, $Y$ might not have the same distributional form as $X_2$.
References:
Kaiser, H. F., & Dickman, K. (1962). Sample and population score matrices and sample correlation matrices from an arbitrary population correlation matrix. Psychometrika, 27(2), 179-182.
Correlation coefficient is the $\cos$ between two series if they are treated as vectors (with $n^{th}$ data point being $n^{th}$ dimension of a vector). The above formula simply creates a decomposition of a vector into its $\cos\theta$, $sin\theta$ components (with respect to $X_1,X_2$).
if $\rho = cos \theta$ ,
then $\sqrt{1-{\rho}^2}=\pm sin \theta$.
Because if $X_1, X_2$ are uncorrelated, the angle between them is a right angle (ie, they can be considered as orthogonal, albeit non-normalized, basis vectors ).
