2
$\begingroup$

I want to predict a variable $Y$ given a set of variables $X_i$. To account for nonlinearity, my $X_i$ are put in several quantile dummies, so that I prefer transforming my $y$.

My $Y$ variable are different kinds of capital income, therefore always positive and skewed with high value variables. My explicative variables are part sociodemographic descriptors, and part other kind of incomes.

If I try to use $$ Y=Xb+u $$

I get this QQ plot, which is not satisfying. enter image description here

If I use a log, $$ ln(Y)=Xb+u $$

I get this one, which is better, but with still some problems at the tails.

enter image description here

Are there any other common transformations ? I have tried with $Y^\alpha$ but it seems mostly worse, although I have not tested a lot of $\alpha$ yet. And specifically are there transformations useful in my case, that is the one in which residuals are still a bit more extreme after a log transformation.

Note that in the end I want to do prediction on another dataset, so I am also interested in the back transformation to get $Y$ back, which I have asked a bit about here

$\endgroup$
  • 3
    $\begingroup$ The criterion here seems to be that you have the right model structure if you can only get the residuals normal. But there is another possibility: you need a quite different kind of model. Tell us about your response or outcome variable. If it is highly skewed and never negative, Poisson regression is likely to be a much better starting place, regardless of whether the response is discrete. The back transformation issue doesn't arise as predictions are returned on the original scale. See also material on generalized linear models. $\endgroup$ – Nick Cox Apr 19 '17 at 9:55
  • $\begingroup$ Why the title flags econometrics is not clear. Nothing mentioned here seems in any sense limited or intrinsic to econometrics. You've tagged transformations on which there are 1.2k questions, so did you check out the most highly voted? $\endgroup$ – Nick Cox Apr 19 '17 at 10:01
  • $\begingroup$ Modifications made according to your suggestions. I have browsed across a few searches and mostly found stuff about log transformations. $\endgroup$ – Anthony Martin Apr 19 '17 at 10:06
  • 3
    $\begingroup$ I'd recommend rather log link. See e.g. blog.stata.com/2011/08/22/… There are various different names for this model. $\endgroup$ – Nick Cox Apr 19 '17 at 10:09
  • 1
    $\begingroup$ I'd also recommend Poisson with sandwich. Please take a look at the book: "Modelling Count Data" by Hilbe. I have a copy and I like it. $\endgroup$ – HelloWorld Apr 19 '17 at 11:09
0
$\begingroup$

I would consider trying a BoxCox transformation if I were in your position. Better explanations for this are provided elsewhere - https://stackoverflow.com/questions/33999512/how-to-use-the-box-cox-power-transformation-in-r.

Failing that, the most common transformations are power and logarithmic transformations (depending on the current type of skewness).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.