Linear regression optimization

Question

I have the following linear model with 2 predictors $x_1, x_2$ and a cubic transform on $x_1$: $$ \hat{Y} = c +\beta_1x_1 + \beta_2x_1^2 + \beta_3x_1^3 + \beta_4x_2 $$

Where $x_1$ can assume negative values. I ran several regressions on subsets of the original population based on the absolute value of $x_1$ and noticed that $R^2$ improves gradually as I increase the minimum required abs value of $x_1$. For example, $R^2$ is higher for the sub-population where abs($x_1) > 10$; sub-population abs($x_1$) > 20 has higher $R^2$ then abs($x_1$) > 10 and so on.

Can I add an interaction between abs($x_1$) and the full set of terms in the model? Or are there linear optimization techniques to solve this?

UPDATE

By imposing a constraint on the absolute value of $x_1$, effectively we are saying the more the information available, or the stronger the signal, the higher the quality of the prediction. I wonder if this is still a case for interaction analysis, or if there are other techniques that can be used more effectively.

Does it make sense to distinguish between the non-linear relationship between $x_1$ and $\hat{Y}$ represented by the cubic transformation (let's call it a "quality relationship"), and an "information quantity" term represented by the absolute value of $x_1$?

Answer 1

The effect you are observing comes about because the true relationship between $Y$ and $x_1$ is not actually a cubic polynomial. As a consequence, the estimation procedure is finding the "closest" cubic polynomial to the true relationship, loosely writing, rather than the true relationship itself. As you shrink the range of $x_1$, though, under a fairly broad range of assumptions, the cubic polynomial will become a better and better approximation to the true relationship (in fact, a linear function will too; this is known as "local linearity".) Consequently, your $R^2$ goes up, because model misspecification error goes down.

You might consider using something other than a cubic polynomial to fit the relationship. Specifically, a spline-based model, as in the R package "mgcv", or a local regression, as in the R package "locfit", provide more flexible modeling than a fixed polynomial functional form can.