# Approximating interactions in OLS

Suppose I have a simple linear model with two variables and their interactions:

$y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_1 x_2 + \beta_4 x_1^2 + \beta_5 x_2^2 + \epsilon$

where the $\beta_i$ can, of course, be estimated by OLS. Assume that the standard Gauss-Markov assumptions of OLS hold, and that data points are sampled i.i.d. My question is the following: Is there a way to approximate this model such that the approximation is separable in $x_1, x_2$?

Clearly, I could just omit the term $\beta_3 x_1 x_2$ and obtain a separable model at the cost of accuracy, but I'm wondering if there is an approach which would better approximate the original model? I'm only interested in predicting $y$, so the metric by which an approximation would be assessed would be the additional error in introduces in predicting $y$. In other words, I'm interested in an approximation which minimizes the difference in sum-squared-residuals relative to OLS.

The approximation need not be linear in its parameters (or even parametric). This seems related to the idea behind partitioned regression. I also know that orthogonal polynomials are commonly applied to function approximation, but again, am not sure if it would be useful here.

If anyone has any suggestions about where to look for some ideas/theory I'd be happy to hear them.

• Your problem will not be sufficiently well defined until you can provide a quantitative way to evaluate how well any approximation works. How are we supposed to measure the goodness of an approximation? The usual methods based on likelihoods or sums of squares are not available because there are no data in evidence. – whuber Sep 18 '17 at 21:59
• Thanks, I've edited my post to clarify the meaning of a "good" approximation. – aht Sep 18 '17 at 23:45
• I don't follow this. Why don't you just include the interaction term & then use OLS? That will do exactly what you want. What, then, is your question? – gung - Reinstate Monica Sep 19 '17 at 0:55
• @gung - there are sometimes computational advantages to having a separable model. If the cost (in terms of accuracy) incurred by using an approximation is not too high then it may be an acceptable price to pay for computational efficiency. – aht Sep 19 '17 at 2:05

You might, for example look into approaches like ACE (alternating conditional expectations -- see Breiman, L. and Friedman, J. H. (1985). "Estimating optimal transformations for multiple regression and correlation.", JASA 80 , 580-78. This is implemented as ace in package acepack for R), for example.
You can do simpler things; if you fit reasonably flexible models in the two predictors (such as via splines for example), you can use Box-Cox transformations (and even estimate the $\lambda$ parameter) to sometimes get reasonable transformations for the response. While less flexible this may be easier to interpret and explain.
• As a simple example consider a relationship (without error) like $y = 9x_1^2 + 6 x_1x_2+x_2^2 + 12x_1+4x_2+8$. That $x_1x_2$ term makes it not-additive; if you model it as $y=f(x_1)+g(x_2)$ you'll have a very distinct pattern in your residuals (a sort of saddle shape). But if you look at $\sqrt{y-4}$, in this case you get a nice linear function in $x_1$ and $x_2$ and zero error. In general no transformation will induce actual additivity of the predictors but you can often get a quite reasonable approximation. – Glen_b Sep 19 '17 at 2:25