1
$\begingroup$

I ran a multivariate regression model in R and got an overall F-statistic of 3,525.690 and three stars attached to it. This seems to be quite good from a statistical point of view.

However, should I be concerned about something or is this really that good?

Improve this question
$\endgroup$
5
  • $\begingroup$ Welcome to Cross Validated! What is your sample size? $\endgroup$
    – Dave
    Commented Apr 26, 2022 at 15:41
  • $\begingroup$ Thanks! I have 946 observations. $\endgroup$
    – TFT
    Commented Apr 26, 2022 at 15:50
  • $\begingroup$ Three stars (i.e. asterisks) is conventionally understood to indicate that $p < 10^{-3}$, where $p$ is the p value in a hypothesis test, but it doesn't hurt to tell us what is being denoted. $\endgroup$
    – Galen
    Commented Apr 26, 2022 at 16:02
  • 1
    $\begingroup$ The statistical point of view doesn't imply that any result is "good" or "bad". The result can be informative either way. We have no idea what your aim of analysis is and therefore we can't tell you whether there is any reason to be happy about this. $\endgroup$
    – Christian Hennig
    Commented Apr 26, 2022 at 16:15
  • $\begingroup$ @ChristianHennig While your comment is correct, my interpretation of the question was if this should suggest a coding error, like you would know you made a mistake if you wound up with variance less than zero. $\endgroup$
    – Dave
    Commented Apr 26, 2022 at 16:20

1 Answer 1

Reset to default
1
$\begingroup$

Let $n$ be the sample size.

Define $RSS$ to be the residual sum of squares for the model, and define $TSS$ to be the total sum of squares. That is:

$$ RSS = \sum_{i=1}^n (y_i - \hat y_i)^2\\ TSS = \sum_{i=1}^n (y_i - \bar y)^2 $$

Now define $p$ to be the number of parameters in the model (including the intercept).

Then: $$F = \dfrac{ \dfrac{ TSS-RSS }{ p-1 } }{ \dfrac{ RSS }{ n-p } } = \dfrac{(TSS-RSS)(n-p)}{RSS(p-1)} $$

If $n$ is large and $p$ is small, then even a small difference between $TSS$ and $RSS$ (so low $R^2$) would correspond to a large $F$-statistic.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Simulation forthcoming $\endgroup$
    – Dave
    Commented Apr 26, 2022 at 15:51
  • $\begingroup$ Thank you so much for your detailed explanation! Actually, I also have a very high adjusted R2 (0.968) together with the high F-statistic. My regression model has 8 variables plus one constant. So, technically, everything looks super fine but it still doesn't mean that the model must be the correct one, right? $\endgroup$
    – TFT
    Commented Apr 26, 2022 at 15:56
  • 1
    $\begingroup$ As George Box said, “All models are wrong, but some are useful.” You seem to have good performance. Is your model useful for what you want to do? $\endgroup$
    – Dave
    Commented Apr 26, 2022 at 16:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.