I want to know if it is possible for a library in R to evaluate the association of independent variables and create a formula? I am trying to come up with a model to predict power consumption of a machine, using some hardware counters and performance attributes. When I use linear regression, I have no problem since I could represent my formula like power~lm(a1+a2+a3+a4), but for the non-linear case, I am not sure what would be the formula or which model should I choose. I would want to have a way to do this:

power ~ <some-non-linear-reg-pkg>(a1+a2+a3+non-linear(a4))

I reviewed some packages for non-linear regression such as nls and gnm, and they expect a formula to be provided by the user. I am however able to identify which variables have linear associations and which are non-linear (by performing correlation tests), the problem is building a formula out of them.

  • $\begingroup$ I have plotted the data, and I don't see a straight relation; can you please elaborate on - "It might give you some inspiration as to the kinds of formula you'd want for a4"? $\endgroup$ – Sayan Mar 29 '13 at 18:29
  • $\begingroup$ Sorry, I thought it might work, but I just tried and it's not even close. I've deleted my previous comment. $\endgroup$ – Wayne Mar 29 '13 at 18:39
  • 2
    $\begingroup$ Because you appear to be in an exploratory mode ("trying to come up with a model"), why not first use exploratory methods like loess smoothing? $\endgroup$ – whuber Mar 29 '13 at 18:41
  • $\begingroup$ nls does non linear least squares in R. It is used differently from lm because the models aren't linear; you can't leave the parameters implicit. They appear in the formula. See ?nls. There's also the possibility of using glms if your model has a linear predictor (i.e. a transformation of the mean of the response is linear in predictors). But I would agree with @whuber - if you don't have a specific functional form in mind, start with some exploratory tools. $\endgroup$ – Glen_b Mar 30 '13 at 1:05
  • $\begingroup$ I tried gam, and I am getting better prediction accuracy. This might be a dumb question - but I am getting this vibe that a linear model is generally sufficient for most cases, may be this is a topic for another question, but comments are welcome. $\endgroup$ – Sayan Mar 30 '13 at 7:50

Look at using linear regression but with a polynomial (poly) function or spline function on the predictors that you think may have a non-linear relationship. Then plotting and further examination of those results may suggest the form of a non-linear function (or the linear model may be sufficient for your purposes).


library(TeachingDemos)  # for Predict.Plot

     fit.lm1 <- lm( Sepal.Width ~ ns(Petal.Width,3)*ns(Petal.Length,3)+Species,

     Predict.Plot(fit.lm1, pred.var = "Petal.Width", Petal.Width = 1.22,
           Petal.Length = 4.3, Species = "versicolor",
             plot.args = list(ylim=range(iris$Sepal.Width), col='blue'),
           type = "response")
     Predict.Plot(fit.lm1, pred.var = "Petal.Width", Petal.Width = 1.22,
           Petal.Length = 4.3, Species = "virginica",
             plot.args = list(col='red'),
           type = "response", add=TRUE)
     Predict.Plot(fit.lm1, pred.var = "Petal.Width", Petal.Width = 1.22,
           Petal.Length = 4.4, Species = "virginica",
             plot.args = list(col='purple'),
           type = "response", add=TRUE)

This is actually the 1st example from the help page for Predict.Plot.


A nonlinear regression function is never going to decide the functional form for you.

You might want to stick with a linear model and use a transformation of the variable with the nonlinear relationship. For example, check the correlation between power $a4^2$, $ln(a4)$, or some other transformation and use which ever one has the higher correlation.

  • $\begingroup$ Thank you, I noticed that using ln in the formula yielded better prediction accuracy. I guess I need to explore the effects of transformations on the independent variables. $\endgroup$ – Sayan Mar 30 '13 at 7:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.