Does it make sense to add a quadratic term but not the linear term to a model?

Question

I have a (mixed) model in which one of my predictors should a priori only be quadratically related to the predictor (due to the experimental manipulation). Hence, I would like to add only the quadratic term to the model. Two things keep me from doing so:

1. I think I read somehwere that you should always include the lower order polynomial when fitting higher order polynomials. I forgot where I found it and in the literature I looked at (e.g., Faraway, 2002; Fox, 2002) I cannot find a good explanation.
2. When I add both, the linear and quadratic term, both are significant. When I add only one of them, they are not significant. However, a linear relation of predictor and data is not interpretable.

The context of my question is specifically a mixed-model using lme4, but I would like to get answers that could explain why it is or why it is not okay to inlcude a higher order polynomial and not the lower order polynomial.

If necessary I can provide the data.

Answer 1

1. Why include the linear term?

It is illuminating to notice that a quadratic relationship can be written in two ways:

$$y = a_0 + a_1 x + a_2 x^2 = a_2(x - b)^2 + c$$

(where, equating coefficients, we find $-2a_2 b = a_1$ and $a_2 b^2 + c = a_0$). The value $x=b$ corresponds to a global extremum of the relationship (geometrically, it locates the vertex of a parabola).

If you do not include the linear term $a_1 x$, the possibilities are reduced to

$$y = a_0 + a_2 x^2 = a_2(x - 0)^2 + c$$

(where now, obviously, $c = a_0$ and it is assumed the model contains a constant term $a_0$). That is, you force $b=0$.

In light of this, question #1 comes down to whether you are certain that the global extremum must occur at $x=0$. If you are, then you may safely omit the linear term $a_1 x$. Otherwise, you must include it.

2. How to understand changes in significance as terms are included or excluded?

This is discussed in great detail in a related thread at https://stats.stackexchange.com/a/28493.

In the present case, the significance of $a_2$ indicates there is curvature in the relationship and the significance of $a_1$ indicates that $b$ is nonzero: it sounds like you need to include both terms (as well as the constant, of course).

Answer 2

@whuber has given a really excellent answer here. I just want to add a small complimentary point. The question states that "a linear relation of predictor and data is not interpretable". This hints at a common misunderstanding, although I usually hear it on the other end ('what is the interpretation of the squared [cubic, etc.] term?').

When we have a model with multiple different covariates, each beta [term] can generally be afforded its own interpretation. For example, if:
$$ \widehat{\text{GPA}}_{college}=\beta_0+\beta_1\text{GPA}_{highschool}+\beta_2\text{class rank}+\beta_3\text{SAT}, $$

(GPA means grade point average;
rank is the ordering of a student's GPA relative to other students at the same high school; &
SAT means 'scholastic aptitude test' a standard, nationwide test for students going to university)

then we can assign separate interpretations to each beta/term. For instance, if a student's high school GPA were 1 point higher--all else being equal--we would expect their college GPA to be $\beta_1$ points higher.

It is important to note, however, that it is not always permissible to interpret a model in this manner. One obvious case is when there is an interaction amongst some of the variables, as it would not be possible for the individual term to differ and still have all else held constant--of necessity, the interaction term would change as well. Thus, when there is an interaction, we do not interpret main effects but only simple effects, as is well understood.

The situation with power terms is directly analogous, but unfortunately, does not seem to be widely understood. Consider the following model:
$$ \hat{y}=\beta_0+\beta_1x+\beta_2x^2 $$ (In this situation, $x$ is intended to represent a prototypical continuous covariate.) It is not possible for $x$ to change without $x^2$ changing also, and vice versa. Simply put, when there are polynomial terms in a model, the various terms based on the same underlying covariate are not afforded separate interpretations. The $x^2$ ($x$, $x^{17}$, etc.) term does not have any independent meaning. The fact that a $p$-power polynomial term is 'significant' in a model indicates that there are $p-1$ 'bends' in the function relating $x$ and $y$. It is unfortunate, but unavoidable, that when curvature exists, the interpretation becomes more complicated, and possibly less intuitive. To assess the change in $\hat{y}$ as $x$ changes, we will have to use calculus. The derivative of the above model is:
$$ \frac{dy}{dx}=\beta_1+2\beta_2x $$ which is the instantaneous rate of change in the expected value of $y$ as $x$ changes, all else being equal. This is not so clean as the interpretation of the very top model; importantly, the instantaneous rate of change in $y$ depends on the level of $x$ from which the change is assessed. Furthermore, the rate of change in $y$ is an instantaneous rate; that is, it is itself continuously changing throughout the interval from $x_{old}$ to $x_{new}$. This is simply the nature of a curvilinear relationship.

Answer 3

@whuber's answer above is right on target in pointing out that omitting the linear term is the "usual" quadratic model is equivalent to saying, "I am absolutely certain that the extremum is at $x=0$."

However, you also need to check whether the software you are using has a "gotcha". Some software may automatically center the data when fitting a polynomial and testing its coefficients unless you turn off polynomial centering. That is, it may fit an equation that looks something like $Y = b_0 + b_2(x - \bar{x})^2$ where $\bar{x}$ is the mean of your $x$s. That would force the extremum to be at $x=\bar{x}$.

Your statement that both the linear and quadratic terms are significant when both are entered needs some clarification. For example, SAS may report a Type I and/or a Type III test for that example. Type I tests the linear before putting in the quadratic. Type III tests the linear with the quadratic in the model.

Answer 4

Brambor, Clark and Golder (2006) (which comes with an internet appendix) have a very clear take on how to understand interaction models and how to avoid the common pitfalls, including why you should (almost) always include the lower-order terms ("constitutive terms") in interaction models.

Analysts should include all constitutive terms when specifying multiplicative interaction models except in very rare circumstances. By constitutive terms, we mean each of the elements that constitute the interaction term. [..]

The reader should note, though, that multiplicative interaction models can take a variety of forms and may involve quadratic terms such as $X^2$ or higher-order interaction terms such as $XZJ$. No matter what form the interaction term takes, all constitutive terms should be included. Thus, $X$ should be included when the interaction term is $X^2$ and $X$, $Z$, $J$, $XZ$, $XJ$, and $ZJ$ should be included when the interaction term is $XZJ$.

Failure to do so may result in an underspecified model that would lead to biased estimates. This may lead to inferential errors.

If this is the case and $Z$ is correlated with either $XZ$ (or $X$) as will occur in virtually any social science circumstance, then omitting the constitutive term $Z$ will result in biased (and inconsistent) estimates of $\beta_0$, $\beta_1$, and $\beta_3$. Although not always recognized as such, this is a straightforward case of omitted variable bias (Greene 2003, pp. 148–149).