1
$\begingroup$

I have a question regarding standardized regression coefficients and the correlation matrix.

I have a two part problem that I am working on, and I need to show that the correlation matrix R is equal to:

$$R = \frac{1}{N-1}X*'X$$

I'm not quite sure how to set this up. I will post the problem I am working on and how far along I have gotten.

Q: To obtain numerically stable computation of regression coefficients it is often a good idea to first standardize all variables as follows:

\begin{align} y_i* &= \frac{y_i-\bar{y}}{s_y} \\ &\text{ and } \\ x_{ij}* &= \frac{x_{ij}-\bar{x}_j}{s_{x_j}}, \\ &(1 \leq i \leq N, 1 \leq j \leq p) \end{align}

where $\bar{x}$ and $\bar{y}$ are the sample means and $s_y$ and $s_{x_j}$ are the sample standard deviations of the corresponding variables. Denote the estimated regression coefficients (called the standardized regression coefficients) for these data by $\hat{\beta_j}*$ $(0\leq j \leq p)$. Note that the $\hat{\beta_j}*$ are unitless and can be compared with each other to assess the relative importance of the $x_j$'s in their strength of linear relationship of $y$ in the presence of other predictor variables.

a) Show that $\hat{\beta}^*_0=0$. Hence we may define the model matrix $X*$ for the standardized data to be an $N \times p$ matrix by excluding the first column of $1$'s; similarly, let $\hat{\beta}^* = (\hat{\beta}_1^*,,...,\hat{\beta}_p^*)'$ to be a $p \times 1$ vector.

b) Show that the correlation matrix $R = (r_{x_jx_k})$ among the $x_j$'s and the correlation vector $ r = (r_{yx_1},...,r_{yx_p})'$ among $y$ and $x_j$ for $j = 1,2,...,p$ are given by

\begin{align} R &= \frac{1}{N-1}X*'X*$ \\ &\text{ and } \\ r &= \frac{1}{N-1}X*'y*' \end{align}

For part a,

\begin{align} Y_i &= \hat{\beta}_0^* + \hat{\beta}_1^*X_{i1}+...+\hat{\beta}_p^*X_{1p}+\hat{e}_i \\ & \\ \frac{y_i-\bar{y}}{s_y} &= (\hat{\beta}_1\frac{s_1}{s_y})\frac{x_i1-\bar{x}_1}{s_1}+...+(\hat{\beta}_p\frac{s_p}{s_y})\frac{x_ip-\bar{x}_p}{s_1} \\ & \\ \frac{y_i-\bar{y}}{s_y} &= (\hat{\beta}_1\frac{s_1}{s_y})\frac{x_i1-\bar{x}_1}{s_1}+...+(\hat{\beta}_p\frac{s_p}{s_y})\frac{x_ip-\bar{x}_p}{s_p} \\ & \\ \hat{\beta}_0 &= \frac{y_i-\bar{y}}{s_y} - (\hat{\beta}_1\frac{s_1}{s_y})\frac{x_i1-\bar{x}_1}{s_1}+...+(\hat{\beta}_p\frac{s_p}{s_y})\frac{x_ip-\bar{x}_p}{s_p} \\ & \\ \hat{\beta}_0 &= (\hat{\beta}_1\frac{s_1}{s_y})\frac{x_i1-\bar{x}_1}{s_1}+...+(\hat{\beta}_p\frac{s_p}{s_y})\frac{x_ip-\bar{x}_p}{s_p} \\ &\quad -\bigg[(\hat{\beta}_1\frac{s_1}{s_y})\frac{x_i1-\bar{x}_1}{s_1}+...+(\hat{\beta}_p\frac{s_p}{s_y})\frac{x_ip-\bar{x}_p}{s_p}\bigg] \end{align}

Therefore, $\hat{\beta}_0 = 0$.

For part b, I'm able to multiply out the matrices $ X*' X*$, but I'm not entirely sure how to interpret the meaning.

$\endgroup$
  • $\begingroup$ What does "part b" of the question ask? So far you haven't said. $\endgroup$ – whuber Oct 15 '14 at 18:45
1
$\begingroup$

As for b, the key thing to remember is that you are dealing with standardized variables. For such variables, correlation is equal to covariance. And sample covariance of standardized n-dimensional vectors $X_1, X_2$ is $\frac{\sum_{i=1}^n x_{1,i}x_{2,i}}{n-1}$, which is 2nd row, 1st column element of $\frac{1}{n-1}X'X$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.