I did a simple regression where the independent variable was the educational background of the mother. The dependent variable was the language skills score of the child.

I switched them around and did another regression to see which coefficients change and which didn't.

The t-test of the gradient was the same for both regressions. I find it difficult to explain why that is the case. The b-value and the standard error, the values that I need to calculate the t-test, were different for each regression. It puzzles me how dividing a different b-value by a different standard error gets the same outcome.

Edit: I have a related but different question now. If I standardize the two variables, so turn them into z-scores, what would the value of the intercept be? I have no idea how to calculate this. I don't see the relation between z-scores and the intercept. Maybe the answer is quite simple but I have to admit that learning about z-scores was a long time ago so maybe I'm missing something very obvious.

  • $\begingroup$ The question is a bit unclear. First, how is mother education expressed? In binary form? And is the score in language continuous? Given that I am very confused how you ran two t-test with them flipped. If you can actually lay out what you did and post the outputs that may increase the chance of getting answers. $\endgroup$ – Penguin_Knight Mar 3 '17 at 19:30
  • $\begingroup$ Thanks, I forgot to mention that. Mother education was categorical, with different education levels as the categories. Language score was continuous. It was part of the assignment, we had to run a regression with mother education as independent variable and language score as dependent. Then we had to do it again with the variables switched around, so mother education as dependent and language score as independent. We have to look at the coefficients tables SPSS produces for both regressions and report which values change and which don't, and why. $\endgroup$ – user137130 Mar 3 '17 at 19:33
  • $\begingroup$ As to my additional question, I think that the intercept is 0? because z-scores have a mean of 0 and then the intercept would be: mean Y - gradient * mean X and the outcome would be 0 if both mean Y and mean X are 0 now. $\endgroup$ – user137130 Mar 4 '17 at 1:47

$\newcommand{\Cov}{\operatorname{Cov}}\newcommand{\Var}{\operatorname{Var}}$In a simple regression, with centered regressions for simplicity,

$$y_i = bx_i + u_i, i=1,\ldots,n$$

Ordinary Least Squares estimation gives

$$\widehat b = \frac { \widehat \Cov(x,y)}{{\widehat \Var(x)}}$$

While $$\widehat \Var(\widehat b) = \frac {\widehat \sigma^2_u}{\widehat \Var(x)}$$

So in reality the t-statistic is

$$t(\widehat b) = \frac{\widehat \Cov(x,y)}{\widehat \sigma_u\cdot \sqrt{\widehat \Var(x)}}$$

In the reverse model

$$x_i = b'y_i + u'_i, i=1,\ldots,n$$

we will have

$$t(\widehat b') = \frac{\widehat \Cov(x,y)}{\widehat \sigma_{u'}\cdot \sqrt{\widehat \Var(y)}}$$

So what one has to examine is whether

$$\widehat \sigma^2_u\cdot \widehat \Var(x) =\text{??}\;\; \widehat \sigma^2_{u'}\cdot \widehat \Var(y)$$

Switching to matrix representation, these estimated quantities are sums of squared terms divided by some factor ($n$ or $n-1$). Ignore these factors to get

$$(y'M_xy)\cdot (x'x) =\text{??}\;\; (x'M_yx)\cdot (y'y)$$

where $M_x = I-x(x'x)^{-1}x' = I - \frac {xx'}{x'x}$, and analogously for $M_y$. Expanding we examine whether

$$\left[y'\left(I - \frac {xx'}{x'x}\right)y\right]\cdot (x'x) =\text{??}\;\; \left[x'\left(I - \frac {yy'}{y'y}\right)x\right]\cdot (y'y)$$

$$\implies (y'y)(x'x) - (y'x)(x'y) = \text{??} \;\; (x'x)(y'y) - (x'y)(y'x)$$

Each term in parenthesis is a scalar. So indeed the two sides are equal, and therefore what the OP obtained is not approximate or a chance event, it is exact and an algebraic property of the OLS estimation in the simple regression:

$$t(\widehat b) = t(\widehat b')$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.