# In R, how can I transform to normalize residuals when I have a U-shaped Q-Q plot?

I am running a two-way ANOVA with one random variable. My histogram of the residuals is showing considerable (negative?) skew:

And my Q-Q plot of the residuals shows a corresponding U-shaped curve:

Can someone suggest an appropriate transformation to normalize these residuals? I have tried log and sqrt transform, but these aren't doing the trick - Q-Q plots look essentially identical.

Here's the residuals just in case anyone needs them:

resids<-c(0.0907115234404372, -0.00228847655956277, -0.0267884765595627,
-0.0859262641306289, -0.0212262641306287, -0.0667262641306288,
-0.0593952037730363, 0.0185047962269638, -0.0364802818046719,
-0.0345802818046719, -0.0636802818046718, 0.0459197181953281,
0.110319718195328, 0.00411971819532808, 0.0156218781924411, 0.0877218781924409,
-0.0467325370980458, -0.0373134598697733, 0.0841865401302266,
-0.0560134598697732, 0.150586540130227, -0.0111134598697733,
0.152686540130227, -0.045845932936565, -0.0389459329365651, -0.0163459329365649,
-0.00584593293656499, -0.015545932936565, 0.00425406706343501,
0.0194540670634349, -0.0566459329365649, -0.0370459329365651,
0.0271762555042805, -0.0518237444957197, -0.0459770706142297,
-0.0256237444957197, 0.105910766841122, 0.0239136558615476, 0.0456136558615476,
-0.0437396702569626, -0.00593967025696251, -0.0071396702569626,
-0.0236257538123323, 0.0136019650946468, -0.0111257538123324,
-0.00702575381233239, -0.0239257538123323, -0.0280284968467248,
0.0705715031532752, -0.0504829121372117, -0.0429829121372116,
-0.0193829121372118, -0.0509829121372116, 0.10563489248827, -0.0311651075117305,
-0.0259651075117306, 0.0650348924882695, 0.0251893077787564,
-0.00731069222124359, -0.0845911986230001, 0.0188349345620156,
-0.00856506543798452, -0.0402650654379846, -0.00956506543798441,
-0.0695650654379845, 0.0615349345620155, -0.0495650654379844,
0.0476936955538023, -0.0414063044461976, 0.0333936955538023,
-0.0451063044461977, -0.0531413637369638, -0.045694689855474,
0.000385664780562678, 0.0940389908990729, -0.0195610091009271,
-0.0254143352194374, -0.0284143352194373, 0.000298786199770751,
-0.0942012138002293, -0.0013012138002293, 0.00800940250009097,
-0.0193946496405897, 0.0119330692663895, 0.0370053503594103,
-0.0113669307336104, -0.0499459456813103, -0.0415459456813103,
-0.0229459456813104, -0.0300459456813102, 0.0462540543186898,
0.0443540543186898, 0.0407540543186897, 0.0183540543186897, 0.0571540543186897,
0.0363632617174681, -0.0610367382825319, 0.0981632617174681,
0.0474632617174682, -0.0485367382825319, -0.0585367382825319,
0.00866326171746823, 0.0131632617174682, 0.190063261717468, -0.0370367382825318,
0.0362590810548065, 0.0105590810548064, -0.0385409189451935,
0.0369590810548064, 0.0492590810548064, -0.0298409189451936,
-0.0589915546215696, -0.00219155462156961, -0.0737915546215695,
0.0181084453784304, -0.0104915546215696, 0.0627084453784306,
-0.0674915546215695, -0.0622915546215694, -0.0274840518137136,
-0.0470117707206927, 0.0133159481862863, 0.0356882292793073,
-0.0581584446021826, -0.0386584446021827, 0.0445783565165312,
-0.0369216434834687, -0.00882164348346892, 0.0411250303980211,
-0.0485216434834688, -0.021339528891233, 0.0656387579342828,
-0.0313612420657172, -0.023839528891233, -0.0142395288912331,
-0.029639528891233, -0.0310499966187923, 0.0433500033812078,
0.0617500033812077, -0.0366499966187923, -0.0621499966187924,
0.0211500033812078, -0.0632499966187923, 0.0631500033812076,
0.0819500033812077, -0.000349996618792314, -0.0129198942139448,
-0.0450198942139448, 0.130180105786055, 0.138980105786055, -0.0668198942139449,
-0.065919894213945, -0.0167652600956127, -0.00446526009561277,
0.0310347399043871, 0.0220564530788714, 0.0179564530788714, 0.0407347399043871,
0.0111347399043871, 0.0776564530788715, -0.00805388766299564,
-0.0662538876629957, -0.0307083029534827, 0.0541461123370044,
0.113446112337004, -0.0603320519387343, -0.0217320519387341,
-0.0915320519387341, 0.113074320307902, -0.0297256796920982,
0.0209743203079018, -0.0236256796920982, 0.0587743203079019,
-0.0662256796920981, -0.0534256796920982, -0.0338256796920982,
-0.083325679692098, 0.00277707299336427, -0.0294229270066357,
0.0124770729933641, -0.0533229270066358, 0.146077072993364, 0.0317770729933642,
-0.0685448001543059, -0.0316448001543057, -0.0575448001543057,
0.146509615136181, 0.014109615136181, -0.0747903848638189, -0.0166777624015848,
0.0308222375984151, 0.0914222375984151, 0.00242223759841509,
-0.0389777624015848, -0.000977762401584981, 0.0269222375984151,
-0.0204624068697834, 0.102637593130217, -0.0698624068697833,
-0.0742624068697832, -0.0477624068697833, -0.00586240686978323,
0.0955375931302167, 0.0890642743432923, 0.180464274343292, -0.044757438831192,
-0.0723357256567077, -0.0680357256567077, -0.00925287584284296,
-0.0209072911333299, -0.0347528758428428, 0.204647124157157,
0.113207730323683, 0.00190773032368274, -0.0242922696763173,
-0.0405922696763172, -0.0265922696763172, 0.0974077303236829,
-0.0243922696763172, -0.0630922696763172, 0.0283077303236827,
0.0582681073945803, -0.0279318926054197, -0.0236318926054198,
-0.0272318926054196, -0.0347318926054196, -0.0801318926054198,
-0.0626768408136609, 0.0253502995705317, -0.0571497004294683,
0.200850299570532, -0.0440497004294684, -0.0357497004294682,
0.0217502995705317, -0.0505861289667355, 0.0888138710332644,
-0.0749861289667355, 0.0619138710332645, 0.0599138710332645,
-0.0103861289667355, -0.0159861289667356, 0.00541387103326452,
-0.0678160116680873, 0.0365839883319128, 0.0577839883319127,
0.0136839883319126, 0.0878839883319127, 0.0160839883319128, 0.144683988331913,
-0.0139160116680872, -0.0400766221886237, -0.0801766221886238,
-0.0895766221886236, -0.00807662218862371, 0.0238233778113763,
-0.0527766221886237, -0.0292766221886236, 0.00982337781137632,
-0.00237662218862367, -0.0566766221886237, 0.0823492874218354,
0.0203948721313485, -0.0382507125781646, -0.110305127868652,
-0.0752051278686516, 0.153894872131348, 0.100494872131349, 0.0305948721313485,
-0.0123051278686515, -0.0669051278686517, -0.0574913489020565,
0.107108651097943, 0.0480086510979434, 0.0452086510979433, -0.101691348902057,
0.00680865109794326, 0.0692086510979433, -0.0405913489020566,
0.0349086510979433, 0.0499356114062059, -0.0297861017682783,
-0.0142861017682785, 0.0234138982317216, 0.0147356114062058,
0.0696356114062058, -0.0285643885937941, 0.0578818302164203,
-0.0486181697835797, -0.0167181697835797, 0.0221818302164203,
-0.0665338718584472, 0.00596612814155284, 0.0139661281415528,
-0.0721338718584472, -0.0639338718584472, 0.119566128141553,
-0.0489338718584471, -0.0740338718584472, 0.00296612814155295,
-0.0787409112120181, -0.0295409112120182, -0.0417409112120182,
0.0827531906259826, -0.0251468093740175, 0.0166531906259826,
0.00825319062598262, -0.0569468093740175, -0.0354468093740175,
0.0578531906259825, 0.0127531906259826, -0.0787468093740173)

• Would you provide the original data so that it can be transformed, please? The residuals do not help with that. – Carl Oct 21 '16 at 17:26

The boxcox function in the MASS package will give an appropriate range of $\lambda$ values for the Box-Cox transformation. If you have not read the paper by Box and Cox, then you should.

Combine the suggested range with knowledge about the science that generated the data and some common sense to decide on a final value (don't just use $\lambda=0.413$ because that gives the best answer, if the confidence interval includes $0.333$ and $0.5$ then look to see if a square root or cube root makes sense with the science and use the one that makes the most sense).

The log transformed data looks not bad:

trans <- log10(resids-1.0001*min(resids))
qqnorm(trans)
qqline(trans)


If that is appropriate depends on the science behind the data.

• Thanks Roland, but as I understand it, I need to perform the transformation on the response variable rather than on the residuals. Unfortunately, log transformation on the response variable (even the variant that you used) still results in a U-shaped curve of the residuals in the Q-Q plot. BTW, what is the -1.0001*min(resids) part of your log transformation intended to do? – Luke Sep 26 '12 at 14:47
• @Luke That was not clear to me from your question. You might want to look at lme4::lmer or nlme::lme and their family argument. Substracting a bit more than the min avoids negative or zero values when taking the log. – Roland Sep 26 '12 at 15:24
• @Luke While you're correct that the transformation is applied to the response values, since you don't supply the response (nor the factors), there's nothing we can do. You don't even show what it looks like after taking the transformations you mention. We have no way to guess what the issue might be. – Glen_b Oct 21 '16 at 1:56
• This plot is almost as curved as the original, but in the other direction, showing the log was too strong. – whuber Oct 21 '16 at 13:29

In my opinion, the reason why even you tried transformation, is that the Q-Q plot doesn't match a normal distribution. For example, as can be seen on the histogram, the gap between max and min is quite large. That is, the data set has a large big variance. How about trying another distribution? For example, the Weilbull distribution might be useful as it can flexibly fit data.

• The variance is not relevant to the question, because it carries no information about the shape of the distribution of residuals. The implication of using a normal QQ plot is that the regression assumes the residuals are approximately Normal. What, then, would it mean to "try another distribution"? Suppose you were to discover that the residuals indeed appear to be Weibull? What would you do about the original regression? – whuber Oct 21 '16 at 13:28
• The Q-Q plot shows how well the Theoretical Quantiles and Sample Quantiles are match. As you can see on that plot, the left side and right side of the plot seem so apart from the line, It is because of too lower or too large value of the data. So I meant, if I assume that data is arise from the Weibull distribution, That the Q-Q plot test show how well the Theoretical Quantiles (from now on It's Weilbull's quantile) and Sample Quantiles are match. – lisa Oct 25 '16 at 7:44
• You mentioned that a normal QQ plot is that the regression assumes the residuals are approximately Normal, But, I think, as the Luke first, assume the normal distribution, then the QQ plot is used to test does really that data is from Normal distribution. However, the QQ plot is not always mean normal test, It's depends on the distribution what you assume. Also, I think to test whether it's Normal distribution using residual is Residual verses Fitted or Residuals verse Leverage Plot. – lisa Oct 25 '16 at 7:49