I'm using Excel and R to do this. I'm new to R.

I have a dataset of energy savings arising from capital investments supported by government grants. I've done a multiple linear regression of energy savings against grant size, total investment cost, and several dummies about investment type. The data is far from normal and looks like it needs transformation.

I did a Box-Cox test on y (ie energy savings), resulting in lambda of -0.14, so I transformed y using a natural log. Then, thinking the explanatory variables also needed transforming, I followed this (where it speaks about transformations of predictors), and the lambdas of cost and grant are 2.25 and 1.75 respectively (both have significant p-values), suggesting a X^2 transformation. Additionally (ie as an alternative), I did log transformations on both grant and cost as I had a hunch that would normalise them.

My questions are as follows.

Question 1: Using transformed explanatory variables

So, looking solely at the adjusted R^2 and not removing any non-significant predictors, the adj-R^2 values are as follows.

  1. model 1 (no transformation): 0.5596
  2. model 2 (y = ln(y)): 0.6070
  3. model 3 (y=ln(y), x=x^2 where not dummy): 0.5478
  4. model 4 (y=ln(y), x=ln(x) where not dummy): 0.6809

I think I should use the ln transformation (ie model 4), but the box.tidwell lambdas recommended a x^2 transformation.

Am I justified using a ln transformation on the explanatory variables despite what the box.tidwell test says? Why? This page suggests I can compare models in R with anova(fit1, fit2), but I don't really know how to evaluate the output (the smaller RSS is better?).

Question 2: Addressing multicollinearity

While doing all this, I realised I had forgotten about multicollinearity. Grant size and investment cost are highly correlated. I want to use the model to estimate energy savings for various grant sizes, so I think it's okay that they are correlated. But I was wondering, can I transform the grant column to be something like 'percentage granted'? I suspect that would remove the correlation, but might do something weird because it would be a constrained value (ie always < 1). Or maybe it could make it a dummy or something?

How can I deal with the multicollinearity of grant size and project cost? Is it just a matter of trying different transformations and seeing how they affect the fit?

Thanks so much.

  • 1
    $\begingroup$ The main reason for transforming here should be that way you get closer to linearity. Normality of any marginal distribution is not a problem if you get it, but it is not an assumption for regression. Every good text on regression explains this point. That aside, you are asking for advice on how to model the data, but you never show the data either in listings or in graphs. $\endgroup$ – Nick Cox Jan 2 '18 at 12:10
  • 1
    $\begingroup$ (1) Adjusted $R^2$ may be useful for comparing transformations of predictors--at least if you adjust it for the additional parameters implied by the freedom to transform the predictors--but it is meaningless for comparing transformations of the responses. (2) Don't use Box-Cox parameters like $2.25$ or $1.75$. Indeed, be very suspicious of any estimate of $\lambda$ that lies beyond the interval $[-1,1]$ unless theory suggests it should. (3) Maximum likelihood makes strong assumptions; its Box-Cox estimates can be poor. (4) For a robust approach, see stats.stackexchange.com/a/35717/919. $\endgroup$ – whuber Jan 2 '18 at 13:39

For question 1, I would suggest not transforming variables based on statistical considerations but on substantive ones. Presumably, energy savings is in dollar terms (or some other currency). It often makes sense to take the log of this sort of variable since we often think of money in multiplicative terms rather than additive ones. For instance, the difference between a \$10,000 house and a \$12,000 one is large; the difference between a \$1,000,000 house and a \$1,002,000 house is rounding error. Whether this is the case in your situation isn't completely clear to me, but it seems possible.

If you decide to not transform, based on the above, note that OLS regression does not assume that the variables are normally distributed; it assumes that the errors (estimated by the residuals) are normally distributed. If you find that this assumption is violated, then you can use methods that don't assume linearity of the residuals (e.g. robust regression, quantile regression).

  • 1
    $\begingroup$ Peter: Linearity of the residuals is just a typo here for normality of the errors, but not quite trivial errors should be corrected by the OP! $\endgroup$ – Nick Cox Jan 2 '18 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.