1
$\begingroup$

For linear regression, suppose we know the true, theoretical coefficients of the predictors (say, for a simulation) and the standard deviation of the error term (sigma). For instance, suppose we know the true relationship is:

y=5 + 4 * x1 + 8 * x2 + error, where error is N(0, variance = 4)

If we are given n values for the predictors x1, x2. We would also know the theoretical variance of the estimated coefficients, and can also obtain the theoretical confidence interval for the coefficients and predictions.

But what about R squared and the F-statistic? Is there a way to calculate the "theoretical" or expected value for those given that we know the real coefficients and sigma? How is R squared and F-statistics distributed? For instance, we know for any given coefficient, it is normally distributed with the mean corresponding to the true value, and the standard deviation depends on sigma.

Thanks

$\endgroup$
2
  • $\begingroup$ "If we are given n values for the predictors x1, x2." So you know the values of the regressors? $F$ & $R^2$ depend on the distribution of $\bf X$, so if you do, then yes, otherwise, no. $\endgroup$ Jun 28, 2021 at 18:37
  • $\begingroup$ I find this hard to understand. What about the Y values, which are unknown? If you know the X values only but not the Y values, how would you then be able to estimate R square? You mention a "theoretical R square". Do you mean the one for "the population" or for the true data generating process? I see how this could be calculated from the equation given, but only if I would know the true variances and correlation of the X variables (in the population or DGP). But because you mention that n values of the X variables are known, I guess this is a small set of all possible "population" values. $\endgroup$
    – BenP
    Apr 4 at 21:21

1 Answer 1

0
$\begingroup$

In this very artificial situation, the "F" statistic has a scaled chi-squared distribution, because sigma is known. R squared can be written as a function of the F statistic, so it has a related distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.