1
$\begingroup$

Can anyone point to an online explanation of why the F-test works for multiple regression?

I found tons online, but I am looking for the sometimes contradictory requirement of the explanation being mathematical and intuitive....let's say "geometric" (but I don't want to rule out non-geometric explanations either, if they help). The closest I've gotten to this is James D. Hamilton's "Time Series Analysis", a section in Chapter 8 entitled "F Tests ABout Beta Under Assumptions 8.1(a) Through (c)". The assumptions are: (a) the predictors are deterministic; (b) the residuals are zero-mean and constant-variance; and (c) the residuals are gaussian. However, the resulting expression is quite complex. Even if I keep in mind that a vector of constant betas from regression would yield a simplified form of his H0, I still cannot see how his F ratio corresponds to the more commonly presented mean-square-of-model over mean-square-error.

For context (otherwise, the request for intuitiveness could mean just about anything), I've been reading up on stats & linear algebra, so I get the idea of the residuals being a projection of the dependent variable data onto a space that is orthogonal to the subspace spanned by the design matrix (thank you wikipedia contributors) and that projection is done via "idempotent" matrices. It makes sense that the dimensionality of the subspace for the predictions is the number of predictors, and that the dimensionality of the subspace for the residuals is number of observations less the number of predictors (though it's impossible to picture for any significant number of observations). I get the idea that the covariance matrix for the design matrix rotates a data cloud so that the coordinate axes align with the eigenvectors, then anisotropically stretches the data points along the axes, then does the reverse rotation to put the transformed data cloud back. I haven't found this pictorial insight to help in seeing the intuition for the F-test, though Hamilton does make use of the diagonlization & anisotropic stretching in his treatment.

I've posted this to:

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.