0
$\begingroup$

I have a response variable, $Z$, for which I'm trying to make a linear model.

Here are some of the fit diagnostics plots:

enter image description here

From the fan-like shape of the residual-vs-predicted value plots, I concluded that a $log_{10}$ transformation should be appropriate, so I transformed $Z$ and got the following plots:

enter image description here

Then I iteratively applied $log_{10}$ to form a new variable $W = log_{10}(1+log_{10}log_{10}Z)$ and got this:

enter image description here

The lack-of-fit $p$ value also improves: from $0.01$ with no transformations, to $0.08$ for the last one.

Now, my questions is as follows:

  1. Is it a good practice to iteratively apply transformations until variance is stabilized enough, and $p$ value of the lack-of-fittests improves over $0.05$?
  2. Would it be acceptable to proceed with the analysis of the linear model for $W$, or it would be better to try to fit nonlinear models to $Z$?
$\endgroup$
  • $\begingroup$ If you simply care about prediction and not interpretation and you have validated your data against a validation dataset, I see nothing wrong with this approach. $\endgroup$ – StatsStudent May 21 '16 at 19:00
3
$\begingroup$

Is it a good practice to iteratively apply transformations until variance is stabilized enough, and $p-$value of the lack-of-fittests improves over 0.05?

This practice looks fishy to me. Fundamentally it seems like you need to know what you are expecting in terms of functional form before you start messing around with fits too much. If your expectation is that the response will follow this log of (A + B log of log of Z) functional form then I suppose you should have started with that. If you didn't then it seems like you have a bad fit and you should reflect on why it isn't working.

By continuing to try to make things fit by adding a transformation (assuming you don't have a clear justification for it) you are making the p-value more difficult to interpret. (I think this would essentially be p-value hacking)

Would it be acceptable to proceed with the analysis of the linear model for $W$, or it would be better to try to fit nonlinear models to $Z$?

If you are interested in the behavior of $Z$ I think it would be better to try to fit non-linear models to $Z$. This is unless you expect the functional form of $W$ for $Z$. If you believe that to best represent $Z$ then you should go with that.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.