4
$\begingroup$

I have the following linear model with 2 predictors $x_1, x_2$ and a cubic transform on $x_1$: $$ \hat{Y} = c +\beta_1x_1 + \beta_2x_1^2 + \beta_3x_1^3 + \beta_4x_2 $$

Where $x_1$ can assume negative values. I ran several regressions on subsets of the original population based on the absolute value of $x_1$ and noticed that $R^2$ improves gradually as I increase the minimum required abs value of $x_1$. For example, $R^2$ is higher for the sub-population where abs($x_1) > 10$; sub-population abs($x_1$) > 20 has higher $R^2$ then abs($x_1$) > 10 and so on.

Can I add an interaction between abs($x_1$) and the full set of terms in the model? Or are there linear optimization techniques to solve this?

UPDATE

By imposing a constraint on the absolute value of $x_1$, effectively we are saying the more the information available, or the stronger the signal, the higher the quality of the prediction. I wonder if this is still a case for interaction analysis, or if there are other techniques that can be used more effectively.

Does it make sense to distinguish between the non-linear relationship between $x_1$ and $\hat{Y}$ represented by the cubic transformation (let's call it a "quality relationship"), and an "information quantity" term represented by the absolute value of $x_1$?

$\endgroup$
3
$\begingroup$

The effect you are observing comes about because the true relationship between $Y$ and $x_1$ is not actually a cubic polynomial. As a consequence, the estimation procedure is finding the "closest" cubic polynomial to the true relationship, loosely writing, rather than the true relationship itself. As you shrink the range of $x_1$, though, under a fairly broad range of assumptions, the cubic polynomial will become a better and better approximation to the true relationship (in fact, a linear function will too; this is known as "local linearity".) Consequently, your $R^2$ goes up, because model misspecification error goes down.

You might consider using something other than a cubic polynomial to fit the relationship. Specifically, a spline-based model, as in the R package "mgcv", or a local regression, as in the R package "locfit", provide more flexible modeling than a fixed polynomial functional form can.

$\endgroup$
  • $\begingroup$ I actually meant the population filter applied to abs($x_1$), as $x_1$ can take negative values. But I guess your answer is still relevant. $\endgroup$ – Robert Kubrick Feb 5 '12 at 4:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.