I am trying to fit a nonlinear regression model in R using nls(). I have a form of the equation I want to fit to:
$$y = (a \times x_{1}^c +b \times x_{2}^d) (x_{3}^e)$$
where the coefficients to be found in regression are a,b,c,d, and e. My data is output from a simulation model where $x_{1}$, $x_{2}$, and $x_{3}$ are all integers from $0$ to $10$, with the condition that $x_{1} + x_{2} + x_{3} \le 10$. $y$ is also integer valued and ranges from $0$ to roughly $1000$. The objective is to fit these data to a rate function that will be used in a Markov Chain.
When I try to fit this regression model directly using nls(), my nlsResiduals plot looks like this:
I know that autocorrelated residuals are problematic, and that non-normal residuals can also be problematic. How can I fix this problem? I was thinking of using transforms on the data like
$$\log(y) = \log((a \times x_{1}^c +b \times x_{2}^d) (x_{3}^e))$$
or
$$y^{1/n} = ((a \times x_{1}^c +b \times x_{2}^d) (x_{3}^e))^{1/n}$$ where $n > 1$. I've noticed if $n$ increases, my autocorrelation graph and QQ-plot look "better" (i.e., more scattered and more normal, respectively).
Both of these seem to correct a lot (but not all) of the autocorrelated residuals, and help to make the residuals more normally distributed. Am I on the right track here, or am I committing some cardinal sin in statistics? Once I settle on a transformation, how can I tell which is best?
Any help, suggestions, or comments are very appreciated.
