Skip to main content
added 237 characters in body
Source Link
Sextus Empiricus
  • 86.6k
  • 6
  • 115
  • 304

$\Sigma$ is the covariance matrix, not the correlation matrix. In the correlation matrix you will see both $\sigma$ and $\sigma_x$ terms.

Intuitively, covariance is the part of variance that both variables share together which is the variance of the $X$ variable. See also the following property of covariance rule for sums of variables:

$$\text{Cov}(aX+bY,cU+dV) = ac \text{Cov}(X,U) +ad \text{Cov}(X,V) +bc \text{Cov}(Y,U) +bd \text{Cov}(Y,V) $$

and

$$ \text{Cov}(Y,X) = \text{Cov}(\eta+\alpha X,X) = \alpha \text{Cov}(X,X) + \text{Cov}(\eta,X) = \alpha \text{Var}(X) $$

where the last equality minimally requires zero correlation between the error term $\eta$ and the independent variable $X$ (independence will be sufficient).

So for the Pearson correlation you will have:

$$\rho_{XY} = \frac{\alpha \sigma_x^2}{\sigma_x\sigma} = \alpha \frac{\sigma_x}{\sigma} $$

which corresponds to your intuition that the correlation gets smaller for larger $\sigma$.


Regarding the question in the title, about the joint distribution, you can not determine $P(X,Y)$ without first defining the joint distribution $P(X,\eta)$. It is not enough information that $\eta$ and $X$ are individually normal distributed. This relates a bit to How to calculate conditional probability when only marginals are known?

$\Sigma$ is the covariance matrix, not the correlation matrix. In the correlation matrix you will see both $\sigma$ and $\sigma_x$ terms.

Intuitively, covariance is the part of variance that both variables share together which is the variance of the $X$ variable. See also the following property of covariance rule for sums of variables:

$$\text{Cov}(aX+bY,cU+dV) = ac \text{Cov}(X,U) +ad \text{Cov}(X,V) +bc \text{Cov}(Y,U) +bd \text{Cov}(Y,V) $$

and

$$ \text{Cov}(Y,X) = \text{Cov}(\eta+\alpha X,X) = \alpha \text{Cov}(X,X) + \text{Cov}(\eta,X) = \alpha \text{Var}(X) $$

where the last equality minimally requires zero correlation between the error term $\eta$ and the independent variable $X$ (independence will be sufficient).

So for the Pearson correlation you will have:

$$\rho_{XY} = \frac{\alpha \sigma_x^2}{\sigma_x\sigma} = \alpha \frac{\sigma_x}{\sigma} $$

which corresponds to your intuition that the correlation gets smaller for larger $\sigma$.

$\Sigma$ is the covariance matrix, not the correlation matrix. In the correlation matrix you will see both $\sigma$ and $\sigma_x$ terms.

Intuitively, covariance is the part of variance that both variables share together which is the variance of the $X$ variable. See also the following property of covariance rule for sums of variables:

$$\text{Cov}(aX+bY,cU+dV) = ac \text{Cov}(X,U) +ad \text{Cov}(X,V) +bc \text{Cov}(Y,U) +bd \text{Cov}(Y,V) $$

and

$$ \text{Cov}(Y,X) = \text{Cov}(\eta+\alpha X,X) = \alpha \text{Cov}(X,X) + \text{Cov}(\eta,X) = \alpha \text{Var}(X) $$

where the last equality minimally requires zero correlation between the error term $\eta$ and the independent variable $X$ (independence will be sufficient).

So for the Pearson correlation you will have:

$$\rho_{XY} = \frac{\alpha \sigma_x^2}{\sigma_x\sigma} = \alpha \frac{\sigma_x}{\sigma} $$

which corresponds to your intuition that the correlation gets smaller for larger $\sigma$.


Regarding the question in the title, about the joint distribution, you can not determine $P(X,Y)$ without first defining the joint distribution $P(X,\eta)$. It is not enough information that $\eta$ and $X$ are individually normal distributed. This relates a bit to How to calculate conditional probability when only marginals are known?

added 237 characters in body
Source Link
Sextus Empiricus
  • 86.6k
  • 6
  • 115
  • 304

$\Sigma$ is the covariance matrix, not the correlation matrix. In the correlation matrix you will see both $\sigma$ and $\sigma_x$ terms.

Intuitively, covariance is the part of variance that both variables share together which is the variance of the $X$ variable. See also the following property of covariance rule for sums of variables:

$$\text{Cov}(aX+bY,cU+dV) = ac \text{Cov}(X,U) +ad \text{Cov}(X,V) +bc \text{Cov}(Y,U) +bd \text{Cov}(Y,V) $$

and

$$ \text{Cov}(Y,X) = \text{Cov}(\eta+\alpha X,X) = \alpha \text{Cov}(X,X) + \text{Cov}(\eta,X) = \alpha \text{Var}(X) $$

where the last equality minimally requires zero correlation between the error term $\eta$ and the independent variable $X$ (independence will be sufficient).

So for the Pearson correlation you will have:

$$\rho_{XY} = \frac{\alpha \sigma_x^2}{\sigma_x\sigma} = \alpha \frac{\sigma_x}{\sigma} $$

which corresponds to your intuition that the correlation gets smaller for larger $\sigma$.

$\Sigma$ is the covariance matrix, not the correlation matrix. In the correlation matrix you will see both $\sigma$ and $\sigma_x$ terms.

Intuitively, covariance is the part of variance that both variables share together which is the variance of the $X$ variable. See also the following property of covariance rule for sums of variables:

$$\text{Cov}(aX+bY,cU+dV) = ac \text{Cov}(X,U) +ad \text{Cov}(X,V) +bc \text{Cov}(Y,U) +bd \text{Cov}(Y,V) $$

and

$$ \text{Cov}(Y,X) = \text{Cov}(\eta+\alpha X,X) = \alpha \text{Cov}(X,X) + \text{Cov}(\eta,X) = \alpha \text{Var}(X) $$

where the last equality minimally requires zero correlation between the error term $\eta$ and the independent variable $X$ (independence will be sufficient).

$\Sigma$ is the covariance matrix, not the correlation matrix. In the correlation matrix you will see both $\sigma$ and $\sigma_x$ terms.

Intuitively, covariance is the part of variance that both variables share together which is the variance of the $X$ variable. See also the following property of covariance rule for sums of variables:

$$\text{Cov}(aX+bY,cU+dV) = ac \text{Cov}(X,U) +ad \text{Cov}(X,V) +bc \text{Cov}(Y,U) +bd \text{Cov}(Y,V) $$

and

$$ \text{Cov}(Y,X) = \text{Cov}(\eta+\alpha X,X) = \alpha \text{Cov}(X,X) + \text{Cov}(\eta,X) = \alpha \text{Var}(X) $$

where the last equality minimally requires zero correlation between the error term $\eta$ and the independent variable $X$ (independence will be sufficient).

So for the Pearson correlation you will have:

$$\rho_{XY} = \frac{\alpha \sigma_x^2}{\sigma_x\sigma} = \alpha \frac{\sigma_x}{\sigma} $$

which corresponds to your intuition that the correlation gets smaller for larger $\sigma$.

Source Link
Sextus Empiricus
  • 86.6k
  • 6
  • 115
  • 304

$\Sigma$ is the covariance matrix, not the correlation matrix. In the correlation matrix you will see both $\sigma$ and $\sigma_x$ terms.

Intuitively, covariance is the part of variance that both variables share together which is the variance of the $X$ variable. See also the following property of covariance rule for sums of variables:

$$\text{Cov}(aX+bY,cU+dV) = ac \text{Cov}(X,U) +ad \text{Cov}(X,V) +bc \text{Cov}(Y,U) +bd \text{Cov}(Y,V) $$

and

$$ \text{Cov}(Y,X) = \text{Cov}(\eta+\alpha X,X) = \alpha \text{Cov}(X,X) + \text{Cov}(\eta,X) = \alpha \text{Var}(X) $$

where the last equality minimally requires zero correlation between the error term $\eta$ and the independent variable $X$ (independence will be sufficient).