Hello I have two problems that sound like natural candidates for multilevel/mixed models, which I have never used. The simpler, and one that I hope to try as an introduction, is as follows: The data looks like many rows of the form
x y innergroup outergroup
where x is a numeric covariate upon which I want to regress y (another numeric variable), each y belongs to an innergroup, and each innergroup is nested in an outergroup (i.e, all the y in a given innergroup belong to the same outergroup). Unfortunately, innergroup has a lot of levels (many thousands), and each level has relatively few observations of y, so I thought this sort of model might be appropriate. My questions are
How do I write this sort of multilevel formula?
Once lmer fits the model, how does one go about predicting from it? I have fit some simpler toy examples, but have not found a predict() function. Most people seem more interested in inference than prediction with this sort of technique. I have several million rows, so the computations might be an issue, but I can always cut it down as appropriate.
I won't need to do the second for some time, but I might as well begin thinking about it and playing around with it. I have similar data as before, but without x, and y is now a binomial variable of the form $(n,n-k)$. y also exhibits a lot of overdispersion, even within innergroups. Most of the $n$ are no more than 2 or 3 (or less), so to derive estimates of the success rates of each $y_i$ I have been using the beta-binomial shrinkage estimator $(\alpha+k_i)/(\alpha+\beta+n_i)$, where $\alpha$ and $\beta$ are estimated by MLE for each innergroup separately. This is has been somewhat adequate, but data sparsity still plagues me, so I would like to use all the data available. From one perspective, this problem is easier since there is no covariate, but from the other the binomial nature makes it more difficult. Does anyone have any high (or low!) level guidance?