# Adding a square root link function to an overdispersed negative binomial GLM

I'm analyzing nematode count data (80 data points) from a randomized block design in which I have two factors with both four levels (Plant and Inoc). The data show heavy overdispersion when analyzed with a glm with poisson-distribution. For one of the plant levels counts are very low (below 100), while for others they almost reach 100000. Therefore I analyzed the data with a Negative-binomial distribution (glm.nb in R MASS-package) which results in limited overdispersion of ~1.5 (Res. def. 89.12 over 60DF). The predicted versus residual plots look fine, but I'm still worried about the amount of overdispersion. I thought it might be worth it to try to change the link function from log to sqrt to see whether that helps. To do this you have to give starting values for your coefficients, like model<-glm.nb(Nem_gramroot ~ Block + Plant + Plant*Inoc, data=data, link = sqrt, start = c(10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10)). I've tried putting in 0's, 1's, 10's an the coefficients of the model based on the log link function. However, I always get the message step size truncated: out of bounds. Any ideas on this? And moreover, any thoughts on using sqrt link functions to reduce overdispersion in negative binomial glms?

I'd avoid the entire problem by applying the Box-Cox power transformation to the raw data.

There's a good explanation of the rationale behind Box-Cox on Wikipedia under "power transformations".

As I understand it, non-normally distributed data can be temporarily transformed into normally distributed data. This opens up the opportunity to use all the statistical tests which are not applicable to non-normally distributed data.

Here's some R code demonstrating the use of Box Cox Transforms http://www.r-bloggers.com/on-box-cox-transform-in-regression-models/

The SAS website has some excellent explanations under "proc transreg".

There's a paper on pareonline.net claiming that many research papers failed to use Box-Cox when they should have. "Improving your data transformations: Applying the Box-Cox transformation" is the title.

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.