I am trying to understand F-tests in general and how to apply one for a particular problem for a data set in "Introduction to Statistical Learning with Applications in R".

The data set is Auto.

I am assessing a multiple linear regression model of regressing mpg onto the other variables, two of which are qualitative. The question is determining whether or not the qualitative variable 'origin' is statistically significant.

fit <- lm(mpg~horsepower+I(horsepower^2)+year+displacement+acceleration+factor(cylinders)+factor(origin),data = Auto)

First of all, how do I determine the number of levels and their meaning of origin? I believe there are three levels to origin and they are American, Japanese, and European, but I don't know if level 1 is Japanese, etc.

In the summary I see coefficient estimates for inputs factor(origin)2 and factor(origin)3. Is this because once we know the value of two (they are each either 0 or 1) we know the third?

So now to test statistical significance of 'origin' we were told to do a partial F-test. How do I do this by hand and using R? How would I know to do such a test in the first place? Would I need to run two: one for factor(origin)2 and one for factor(origin)3?

Does the F-test test significance by testing the hypothesis that the coefficient of factor(origin)2 is 0 and so of no statistical significance?

  • $\begingroup$ You can see the levels doing: levels(as.factor(Auto$origin)) $\endgroup$ – igorkf Sep 19 at 21:07
  • $\begingroup$ When you use lm() with categorical variables, R uses contrasts, a way of representing the occurence or not of these variables in a matrix. By default, R always use the first level of a category in the intercept. This is a way of create a invertible design matrix (the X matrix). So the first level of every categorical variable will not appear in the result. This is learned in Linear Models. You can see the X matrix doing model.matrix(fit) $\endgroup$ – igorkf Sep 19 at 21:11

As noted by @igorkf, the reason that you're only being shown certain level of the factor "origin" is because lm() will by default use dummy coding on unordered factors. To understand the dummy coding, consider a simple example:

$\hat{y} = 15.00 + \beta_x$, where $x$ is a dichotomous factor (say it's gender).

In this case, we can code this dichotomous factor so that $x$ is 0 for one level (say for female) and 1 for the other (so male). It should be fairly easy to see that when a person is female and $x = 0$ then all we're doing is predicting $y$ from the intercept. In contrast, when the person is male and $x = 1$ we are just adding to the intercept whatever $\beta$ is estimated to be. In this case, female ($x = 0$) is our reference level and then the model estimates the effect of being male ($x = 1$). This same logic applies to additional levels (assuming basic dummy coding is used).

As far as getting a better handle on the model output, I'm assuming that you're getting the data from data(Auto) out of the ISLR package. Looking at that data on my end, the "origin" variable is numeric when loaded in initially. Per the documentation, values of 1 = American, 2 = European, and 3 = Japanese. So, you can make the data a bit easier to understand by using the following:

Auto$origin <- factor(Auto$origin, levels = c(1, 2, 3), labels = c("American", "European", "Japanese"))

In this case, you've defined the "origin" variable to be a factor and told R to expect this factor to have levels with values of 1, 2, and 3. The labels part of the argument then tell R to call those levels "American", "European", and "Japanese", respectively (note that the order of labels must match the order of levels). This should help your output from summary(fit) be a little easier to see, and it means that you don't have to use + factor(origin) and can just do + origin in the lm() function.

Finally, to your point about a partial F-test. My understanding of the partial F-test is that all you're doing is testing the relative contribution of a variable (or group of variables) to a simpler model. In my field, we usually just refer to this as hierarchical regression. The basic idea would be that you have some model predicting a specific outcome. This model has several different predictors that are maybe well-known, commonly accepted predictors, or just possible sources of noise in the sample (that you may want to control for). Say we have some theory that tells us that another predictor is important. We could just see whether or not that significantly predicts the outcome on its own, and that could be useful; however, stronger evidence of this predictor's relevance would be if we could show that it adds to the predictive accuracy of this other model (i.e., that it remains significant even if we include it in a model with other significant predictors).

In R, this test of model improvement would be checked like this:

fit_reduced <- lm(mpg ~ horsepower + I(horsepower^2) + year + displacement + acceleration + factor(cylinders), data = Auto)
fit_full <- lm(mpg ~ horsepower + I(horsepower^2) + year + displacement + acceleration + factor(cylinders) + origin, data = Auto)
anova(fit_reduced, fit_full)

The result of the anova() function is the partial F-test and tells you whether or not the addition of the "origin" factor improved the model fit. If you wanted to do this by hand, then you could use the following formula:

$F = {\frac{SSE_{R}-SSE_{F}}{df_R-df_F}}\div{\frac{SSE_F}{df_F}}$, where $SSE$ is the sum of squared errors, $df$ is degrees of freedom, and the subscript $R$ and $F$ refer to the reduced and full models, respectively.

| cite | improve this answer | |
  • $\begingroup$ Where did you find the documentation for the Auto package to see that 1 = American? $\endgroup$ – E2R0NS Sep 20 at 5:25
  • $\begingroup$ I found that information here: rdrr.io/cran/ISLR/man/Auto.html $\endgroup$ – Billy Sep 20 at 23:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.