Add a quadratic random effect to a nonlinear mixed model? How can one add a quadratic random effect to a nonlinear mixed effect model? I've been trying to do this with nlmer without luck. Any tips would be greatly appreciated!
Edit: as diagnosed by Ben, the root problem is that I am trying to shoehorn a numeric variable into the role of a (categorical, non-numeric) grouping variable.
Here's my starting point that works fine:
nlmem <- nlmer(value ~ nfun(x, y0) ~  (y0|site) + (y0|gas:CTG), data=data.plot, start=initial.guesses) 
...and my failed attempts upon changing the formula to include a quadratic random effect:
value ~ nfun(x, y0) ~  (y0|site) + (y0|gas:CTG) + (y0|CTG*CTG)
    Error: Invalid grouping factor specification, CTG * CTG
    In addition: Warning message:
    In Ops.factor(CTG, CTG) : * not meaningful for factors

value ~ nfun(x, y0) ~  (y0|site) + (y0|gas:CTG) + (y0|poly(CTG, 2))
    Error: couldn't evaluate grouping factor poly(CTG, 2) within model frame: try adding grouping factor to data frame explicitly if possible
    In addition: Warning messages:
    1: In mean.default(x) : argument is not numeric or logical: returning NA
    2: In Ops.factor(x, xbar) : - not meaningful for factors

value ~ nfun(x, y0) ~  (y0|site) + (y0|gas:CTG) + (y0|I(CTG^2))
    Error: Invalid grouping factor specification, I(CTG^2)
    In addition: Warning message:
    In Ops.factor(CTG, 2) : ^ not meaningful for factors

CTG is numeric, not a factor. Not sure why the error message is complaining about it being a factor.

class(data.plot$CTG) 
[1] "numeric"


 A: If CTG is numeric you shouldn't be using it as a grouping variable (the right-hand side of a (f|g) random effect specification).  A grouping variable (g) should always be a categorical/factor variable, or something that can meaningfully be coerced to such.
A possible reason for such a confusion is that, for a categorical effect f,


*

*(1|f:g) describes the variation among categories within groups (a single variance parameter);

*(f|g) describes the variation among effects across groups (i.e., a variance-covariance matrix among the baseline group and differences among groups).  If this variance-covariance matrix is restricted to a positive compound-symmetric structure, e.g.


$$
\Sigma = \left( \begin{array}{cccc} \sigma^2 &  \rho \sigma^2 &    \rho \sigma^2 & \ldots \\
                                    \rho  \sigma^2    & \sigma^2 & \rho  \sigma^2& \ldots \\ 
                                    \vdots   & \vdots   & \vdots & \ddots 
                 \end{array} \right)
$$
(with $\rho>0$; notation isn't perfect, but you get the idea) -- this is theoretically possible, but not in lme4 at the moment -- then you would get exactly the same answer.
Traditionally, with models that only allow among-group differences in the intercept, people have gotten used to thinking about (1|f:g) as the way to model an interaction between a categorical (fixed-effect) predictor and a random grouping variable, so the distinction between effects and grouping variables tends to get blurred.
I think you're looking for a random (quadratic) slopes model, i.e. if nfun is a quadratic function a+b*x+c*x^2 then you would use
 value ~ nfun(x, a,b,c) ~ (a+b+c|site)

(although for a polynomial model you could just use a linear mixed model with x and I(x^2) terms).
