Conceptual/Math/brainteaser Question: Multiple Linear Regression

This is kind of a brainteaser and I'm struggling to solve it, any ideas on approaching this would be valued:

I thought about using substitution of the different x and z into the third regression equation to get a value in terms of x and z but I cannot go any further than that.

I know that the formula for F statistic for testing exclusion restrictions is in terms of SSR

And the formula in terms of R squared is:

• As a brainteaser, E, F, L, M are easy, A, C, H, J can be calculated without much difficulty, G can be calculated simply taking into account some of the formulae you state, and $B, D, I, K$ are rather more effort but less interesting. Commented Aug 21, 2021 at 0:44
• Hi Henry, thank you for your response! Could you please further elaborate on how you would go about calculating these values? I'm quite new to these concepts and hence finding it tricky. Commented Aug 21, 2021 at 1:50
• This brain teaser looks rather similar to questions I have seen set for an exam or an assignment. Where does it originate? e.g. do you have a link to the original teaser? Commented Aug 21, 2021 at 2:00
• The basic underlying idea is that the fit depends only on the space generated by the explanatory variables and not on the variables themselves.
– whuber
Commented Aug 21, 2021 at 14:49
• Because the transformation from $(v,w)$ to $(x,z)$ is linear and invertible, models (1) and (2) must have identical fits. Similarly model (4) has the same fit. Therefore they all have the same values of SSR and $R^2.$ The coefficients for model (1) will transform appropriately to give the coefficients for models (2) and (4). As @Henry remarked, that leaves you to compute the standard errors of estimate for various coefficients: for that you need to work out how the variance-covariance matrix transforms.
– whuber
Commented Aug 21, 2021 at 18:05

This set of questions probes your understanding of the relationships among (a) the parameter estimates and (b) their variance-covariance matrix when the explanatory variables in an ordinary least squares (OLS) regression are transformed. Let's begin, then, by discussing this generally. And because this is a self-study question, I won't provide the specifics: that's for you to enjoy working out.

Let the original model be

$$E[y] = X\beta$$

where the columns of the model matrix $$X$$ are the values of the explanatory variables (perhaps including a constant, or "intercept," term) and $$\beta$$ is the corresponding vector of (unknown) coefficients to be estimated.

Suppose instead $$X= Z\mathbb{A}$$ expresses the variables in terms of other variables $$Z$$ where the matrix $$\mathbb A$$ represents an invertible linear change of variables with inverse $$\mathbb{A}^{-1}.$$ Because

$$X\beta = X\,\mathbb{A}^{-1}\mathbb{A}\,\beta = (X\mathbb{A}^{-1})\, (\mathbb{A}\beta) = Z(\mathbb{A}\beta),$$

by writing $$\gamma=\mathbb{A}\beta$$ for the new parameters the model is expressed as

$$E[y] = Z\gamma.$$

Consequently, when $$\hat\beta$$ is the OLS estimate of $$\beta$$ with (estimated) variance-covariance matrix $$\widehat{\mathbb{V}} = \operatorname{Var}(\hat\beta),$$ the corresponding estimate of $$\gamma$$ must be $$\hat\gamma = \mathbb{A}\hat\beta,$$ because both of them yield the same sum of squares of residuals. Moreover, the variance-covariance matrix for $$\hat\gamma$$ is

$$\widehat{\mathbb W} = \operatorname{Var}(\hat\gamma) = \operatorname{Var}(\mathbb{A}\hat\beta) = \mathbb{A}\operatorname{Var}(\hat\beta) \mathbb{A}^\prime = \mathbb{A}\widehat{\mathbb{V}}\mathbb{A}^\prime.$$

These observations quickly (and easily) yield all the missing values except (G), the $$R^2$$ for model (3). That can be determined from the $$R^2$$ and $$SSR$$ values in models (1) and (3). The idea is that the data in column (1) suffice to determine the variance of $$y.$$ That and the $$SSR$$ in column (3) yield its $$R^2$$ value.

The hardest part is finding the off-diagonal terms of the variance-covariance matrices. This amounts to determining the covariance of $$v$$ and $$w$$ (or, equivalently, of $$x$$ and $$z$$). Since models (1), (2), and (4) are equivalent and the standard errors (square roots of diagonal elements of the variance-covariance matrices) are given only for (1), there's little hope of finding the correlation from those data alone: thus, the key value must be the standard error of $$v$$ in model (3). To relate it to model (1), use the fact that $$\operatorname{Var}(v) = \operatorname{Var}(x+z) = \operatorname{Var}(x) + \operatorname{Var}(z) + 2\operatorname{Cov}(x,z).$$

Finally, because all models include an intercept, you may first "take out" the intercept (as usual) by centering all the variables. This reduces your models from three (or two) variables to just two (or one). Moreover, the SSP matrix $$X^\prime X$$ will then be a multiple of the covariance matrix of the variables. This matrix plays a prominent role in formulas for the parameter estimates and estimates of their variance-covariance matrix: it is what you want to focus on.

Please notice that because the question concerns only estimates and standard errors of estimate, nothing depends on distributional assumptions. Don't even think about Normal, Student t, or $$F$$ distributions: although you might be able to compute some of the missing values that way, it would be a rather roundabout method (essentially, you would be converting estimates and standard errors to probabilities and then converting back again) and likely quite a bit more complicated than necessary.