Understanding the mixed effect model thru creating my own optimization-based model

Following Macro's suggestion, I am creating a new thread for my questions. (It's somehow related to the original question,

Understanding the linear mixed effects model equation and fitting a random effects model with weights in R

Specifically, I am trying to understand a random intercept model in the form of $${\bf y}={\bf X}{\boldsymbol \beta} + {\bf Z} {\bf b} + {\boldsymbol \varepsilon}$$

(Following Macro's notation for ${\boldsymbol y_i}$ in the previous question, I stack the group formulas together vertically and obtained one big formula for the whole dataset with all groups).

My questions are:

1. In a random intercept model (all group sharing the same beta), will the following situation arise: all data points in a group reside on one side (the same side) of the regression line of that group? Here is my thinking: even though for a certain group, the overall beta is a very crappy one for that group. By adjusting the intercept for that group, I at least still can get the regression line passing the cloud, am I right? Here by "passing the cloud" I mean not all points will reside on the same side of the regression line.

2. In a random intercept model, for each group, which point does the regression line pass thru? In the general OLS, we knew that it's the point $(\overline{x}, \overline{y})$ that is passed thru by the regression line. But for each group in a random intercept model, which point is that "central" point?

3. Moreover, in R, what's a convenient way to visualize what's happening within a certain group in a mixed model?

4. How do I understand the ${\bf b}$'s? What are these ${\bf b}$'s? May I say they are simply the $\overline{y}_i - \overline{y}$, i.e. the vertical axis of the group centers - the vertical axis of the whole-data center?

5. Why does lme explicitly impose that the sum of ${\bf b}$ is $0$ ?

6. To understand the lme models better, I am creating my own toy models and doing some comparative studies:

To cook my own model, I decide to solve the following least-square problem:

$$\min \left( \sum_{k=1}^{K} w_{1k} {\bf b}_k^2 + \sum_{i=1}^{N} w_{2i} \varepsilon_i^2 \right)$$

here $\{ {\bf b}_k \}_{k=1}^{K}$, where $K$ is the number of groups. $\{ \varepsilon_i \}_{i=1}^{N}$, where $N$ is the number of individuals.

$w_1$ is a set of weights on the individuals and $w_2$ is a set of weights on the groups at the group level.

My purpose is to create a model that resembles the random intercept model, but ditching the Gaussian distribution assumption for the ${\bf b}$'s and I would like to have the regression line in each group(sharing the same beta) passing the data cloud for that group (Please ref. Q1 above).

• Do you think my model makes sense?

• Could you please help me critique my model vs. the lme random intercept model?

• Shall I add more constraints such as $\sum {\bf b}_k = 0$and $\sum \varepsilon_i =0$?

Thanks a lot for your help!

• Luna, I tried to fix this up so that it is readable (in its previous state it was extremely difficult to tell what was being asked). I tried to transcribe your estimating equation verbatim but I'm not sure I did it correctly since what is there now is difficult to make sense of - what argument is being minimized over? The $w$s? Do the fixed effects, ${\boldsymbol \beta}$, come into play anywhere? – Macro Jun 25 '12 at 21:06
• Thank you so much Macro for your help! It looks great now! Depending the answer to my Q4, if my understanding of the b's is correct, then I guess I wanted to minimize the sum of the weighted sum of squared b's (if b's are indeed the distance between the overall intercept and the group-wise intercepts) and the weighted sum of the squared (within-group) residuals... – Luna Jun 25 '12 at 23:34
• minimize as a function of...? If it's a function of the ${\bf b}$'s then clearly the answer is to make them all $0$. What are the weights? – Macro Jun 25 '12 at 23:36