3
$\begingroup$

I have a regression model

$y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 +u$

It is known that sample means of both $x_1$ and $x_2$ are zero, moreover the error term is said to be homoskedastic, the standard error of regression, standard errors of OLS estimates of the slope coefficients, and number of observations $n$ are given and is equal to: $se(\hat{\beta_1}) = 1,\ se(\hat{\beta_1}) = 2,\ \hat{\sigma} = 3,\ n = 100$. Can we say something about total sum of squares (i.e. $\sum_{i=1}^n (y_i - \bar{y})^2$), namely, can we somehow estimate its smallest possible value?

What I have tried:

$\sum_{i=1}^n (y_i - \bar{y})^2 = \sum_{i=1}^n(\hat{\beta_0} + \hat{\beta_1}x_{1i} + \hat{\beta_2}x_{2i} + \hat{u_i} - \bar{y})^2 = \sum_{i=1}^n (\hat{\beta_1}x_{1i} + \hat{\beta_2}x_{2i})^2 + \sum_{i=1}^n \hat{u_i}^2$.

We know the second term, since we know the standard error. So it remains to estimate the first one. Standard erorrs of OLS estimates of slope coefficients are

$$se(\hat{\beta_1}) = \sqrt{\dfrac{\sum_{i=1}^nx_{2i}^2}{\sum_{i=1}^nx_{1i}^2\sum_{i=1}^nx_{2i}^2- (\sum_{i=1}^nx_{1i}x_{2i})^2}} \hat{\sigma}$$

$$se(\hat{\beta_2}) = \sqrt{\dfrac{\sum_{i=1}^nx_{1i}^2}{\sum_{i=1}^nx_{1i}^2\sum_{i=1}^nx_{2i}^2- (\sum_{i=1}^nx_{1i}x_{2i})^2}} \hat{\sigma}$$

Since we know $se(\hat{\beta_1}), se(\hat{\beta_2}), \hat{\sigma}$ we can compute the ratio $\dfrac{\sum_{i=1}^nx_{1i}^2}{\sum_{i=1}^nx_{2i}^2}$. Moreover we can express $\sum_{i=1}^nx_{1i}^2$ and $\sum_{i=1}^nx_{2i}^2$ through $(\sum_{i=1}^nx_{1i}x_{2i})^2$. Thus, $$\sum_{i=1}^n (\hat{\beta_1}x_{1i} + \hat{\beta_{2}}x_{2i})^2 = \hat{\beta_1}^2\sum_{i=1}^nx_{1i}^2 + \hat{\beta_2}^2\sum_{i=1}^nx_{2i}^2 + 2 \hat{\beta_1}\hat{\beta_2}\sum_{i=1}^nx_{1i}x_{2i}$$

The idea was to plug the expressions of $\sum_{i=1}^nx_{1i}^2$ and $\sum_{i=1}^nx_{2i}^2$ into the aforementioned expression and try to minimize it by $\sum_{i=1}^nx_{1i}x_{2i}$ treating $\hat{\beta_1}, \hat{\beta_2}$ like constants. But I do not think that it is a right way, since once we change $\sum_{i=1}^nx_{1i}x_{2i}$ we also change $\hat{\beta_1}, \hat{\beta_2}$ because the OLS formulas include $\sum_{i=1}^nx_{1i}x_{2i}$. Plugging formulas for OLS estimates also does not seem to be a good idea since these formulas include sample covariances between $y$ and $x_1$, $x_2$ which we do not know. Here I got stuck.

Could you please give me any hints, how to proceed?

Thanks a lot in advance for any help!

$\endgroup$
  • $\begingroup$ Assuming your (undefined) symbol "$\bar y$" refers to the mean of the $y_i,$ the total sum of squares isn't something under your control: it depends only on the data $y_i$ but not on any of the parameter estimates and therefore has nothing to do with OLS or regression. What, then, are you trying to ask? What the smallest possible value is for any dataset? It's hard to believe that's the intended question because it's obvious that when all the $y_i$ are equal, the total sum of squares is zero (and no sum of squares can be negative). What, then, is your question? $\endgroup$ – whuber Feb 10 at 13:09
  • 1
    $\begingroup$ @whuber Yes, $\bar{y}$ refers to the mean of $y_i$. As I wrote I am given with exact value of standard error of regression, which is not 0, say it is equal to 3, hence all $y_i$ cannot be equal, I suppose. My question is: once I have exact values of standard errors of slope coefficients and exact value of error of regression, can I estimate the total sum of squares? $\endgroup$ – D F Feb 10 at 13:16
  • $\begingroup$ Could you clarify at the outset what information you have, what you don't know, and what you are willing to vary in finding this minimum? $\endgroup$ – whuber Feb 10 at 13:20
  • $\begingroup$ @whuber Ok, I've added the information I have. I have nothing beyond it $\endgroup$ – D F Feb 10 at 13:27
  • $\begingroup$ +1. It seems to me the standard errors add no relevant information, which suggests you consider the relationship between the variance of the response and the variance of the residuals. The reason I think this is that the fit could be excellent yet the SEs of the coefficient estimates can be arbitrarily large depending on how collinear the regressors are. $\endgroup$ – whuber Feb 10 at 13:35

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.