Return to Question

Following Macro's suggestion, I am creating a new thread for my questions. (It's somehow related to the original question,

which had been discussed here:

Understanding the linear mixed effects model equation and fitting a random effects model with weights in R

Specifically, I am trying to understand a random intercept model in the form of $${\bf y}={\bf X}{\boldsymbol \beta} + {\bf Z} {\bf b} + {\boldsymbol \varepsilon} $$

(Following Macro's notation for ${\boldsymbol y_i}$ in the previous question, I stack the group formulas together vertically and obtained one big formula for the whole dataset with all groups).

My questions are:

1. In a random intercept model (all group sharing the same beta), will the following situation arise: all data points in a group reside on one side (the same side) of the regression line of that group? Here is my thinking: even though for a certain group, the overall beta is a very crappy one for that group. By adjusting the intercept for that group, I at least still can get the regression line passing the cloud, am I right? Here by "passing the cloud" I mean not all points will reside on the same side of the regression line.

2. In a random intercept model, for each group, which point does the regression line pass thru? In the general OLS, we knew that it's the point $(\overline{x}, \overline{y})$ that is passed thru by the regression line. But for each group in a random intercept model, which point is that "central" point?

3. Moreover, in R, what's a convenient way to visualize what's happening within a certain group in a mixed model?

4. How do I understand the ${\bf b}$'s? What are these ${\bf b}$'s? May I say they are simply the $\overline{y}_i - \overline{y}$, i.e. the vertical axis of the group centers - the vertical axis of the whole-data center?

5. Why does lme explicitly impose that the sum of ${\bf b}$ is $0$ ?

6. To understand the lme models better, I am creating my own toy models and doing some comparative studies:

To cook my own model, I decide to solve the following least-square problem:

$$ \min \left( \sum_{k=1}^{K} w_{1k} {\bf b}_k^2 + \sum_{i=1}^{N} w_{2i} \varepsilon_i^2 \right) $$

here $\{ {\bf b}_k \}_{k=1}^{K}$, where $K$ is the number of groups. $\{ \varepsilon_i \}_{i=1}^{N}$, where $N$ is the number of individuals.

$w_1$ is a set of weights on the individuals and $w_2$ is a set of weights on the groups at the group level.

My purpose is to create a model that resembles the random intercept model, but ditching the Gaussian distribution assumption for the ${\bf b}$'s and I would like to have the regression line in each group(sharing the same beta) passing the data cloud for that group (Please ref. Q1 above).

• Do you think my model makes sense?

• Could you please help me critique my model vs. the lme random intercept model?

• Shall I add more constraints such as $\sum {\bf b}_k = 0$and $\sum \varepsilon_i =0$?

Thanks a lot for your help!

Following Macro's suggestion, I am creating a new thread for my questions. (It's somehow related to the original question,

which had been discussed here:

Understanding the linear mixed effects model equation and fitting a random effects model with weights in R

Specifically, I am trying to understand a random intercept model in the form of $${\bf y}={\bf X}{\boldsymbol \beta} + {\bf Z} {\bf b} + {\boldsymbol \varepsilon} $$

(Following Macro's notation for ${\boldsymbol y_i}$ in the previous question, I stack the group formulas together vertically and obtained one big formula for the whole dataset with all groups).

My questions are:

1. In a random intercept model (all group sharing the same beta), will the following situation arise: all data points in a group reside on one side (the same side) of the regression line of that group? Here is my thinking: even though for a certain group, the overall beta is a very crappy one for that group. By adjusting the intercept for that group, I at least still can get the regression line passing the cloud, am I right? Here by "passing the cloud" I mean not all points will reside on the same side of the regression line.

2. In a random intercept model, for each group, which point does the regression line pass thru? In the general OLS, we knew that it's the point $(\overline{x}, \overline{y})$ that is passed thru by the regression line. But for each group in a random intercept model, which point is that "central" point?

3. Moreover, in R, what's a convenient way to visualize what's happening within a certain group in a mixed model?

4. How do I understand the ${\bf b}$'s? What are these ${\bf b}$'s? May I say they are simply the $\overline{y}_i - \overline{y}$, i.e. the vertical axis of the group centers - the vertical axis of the whole-data center?

5. Why does lme explicitly impose that the sum of ${\bf b}$ is $0$ ?

6. To understand the lme models better, I am creating my own toy models and doing some comparative studies:

To cook my own model, I decide to solve the following least-square problem:

$$ \min \left( \sum_{k=1}^{K} w_{1k} {\bf b}_k^2 + \sum_{i=1}^{N} w_{2i} \varepsilon_i^2 \right) $$

here $\{ {\bf b}_k \}_{k=1}^{K}$, where $K$ is the number of groups. $\{ \varepsilon_i \}_{i=1}^{N}$, where $N$ is the number of individuals.

$w_1$ is a set of weights on the individuals and $w_2$ is a set of weights on the groups at the group level.

My purpose is to create a model that resembles the random intercept model, but ditching the Gaussian distribution assumption for the ${\bf b}$'s and I would like to have the regression line in each group(sharing the same beta) passing the data cloud for that group (Please ref. Q1 above).

• Do you think my model makes sense?

• Could you please help me critique my model vs. the lme random intercept model?

• Shall I add more constraints such as $\sum {\bf b}_k = 0$and $\sum \varepsilon_i =0$?

Thanks a lot for your help!

Following Macro's suggestion, I am creating a new thread for my questions. (It's somehow related to the original question,

which had been discussed here:

Understanding the linear mixed effects model equation and fitting a random effects model with weights in R

Specifically, I am trying to understand a random intercept model in the form of $${\bf y}={\bf X}{\boldsymbol \beta} + {\bf Z} {\bf b} + {\boldsymbol \varepsilon} $$

(Following Macro's notation for ${\boldsymbol y_i} in the previous question, I stack the group formulas together vertically and obtained one big formula for the whole dataset with all groups).

My questions are:

1. In a random intercept model (all group sharing the same beta), will the following situation arise: all data points in a group reside on one side (the same side) of the regression line of that group? Here is my thinking: even though for a certain group, the overall beta is a very crappy one for that group. By adjusting the intercept for that group, I at least still can get the regression line passing the cloud, am I right? Here by "passing the cloud" I mean not all points will reside on the same side of the regression line.

2. In a random intercept model, for each group, which point does the regression line pass thru? In the general OLS, we knew that it's the point $(\overline{x}, \overline{y})$ that is passed thru by the regression line. But for each group in a random intercept model, which point is that "central" point?

3. Moreover, in R, what's a convenient way to visualize what's happening within a certain group in a mixed model?

4. How do I understand the ${\bf b}$'s? What are these ${\bf b}$'s? May I say they are simply the $\overline{y}_i - \overline{y}$, i.e. the vertical axis of the group centers - the vertical axis of the whole-data center?

5. Why does lme explicitly impose that the sum of ${\bf b} is $0$ ?

6. To understand the lme models better, I am creating my own toy models and doing some comparative studies:

To cook my own model, I decide to solve the following least-square problem:

$$ \min \left( \sum_{k=1}^{K} w_{1k} {\bf b}_k^2 + \sum_{i=1}^{N} w_{2i} \varepsilon_i^2 \right) $$

here $\{ {\bf b}_k \}_{k=1}^{K}$, where $K$ is the number of groups. $\{ \varepsilon_i \}_{i=1}^{N}, where $N$ is the number of individuals.

$w_1$ is a set of weights on the individuals and $w_2$ is a set of weights on the groups at the group level.

My purpose is to create a model that resembles the random intercept model, but ditching the Gaussian distribution assumption for the ${\bf b}$'s and I would like to have the regression line in each group(sharing the same beta) passing the data cloud for that group (Please ref. Q1 above).

• Do you think my model makes sense?

• Could you please help me critique my model vs. the lme random intercept model?

• Shall I add more constraints such as $\sum {\bf b}_k = 0$and $\sum \varepsilon_i =0$?

Thanks a lot for your help!

Following Macro's suggestion, I am creating a new thread for my questions. (It's somehow related to the original question,

which had been discussed here:

Understanding the linear mixed effects model equation and fitting a random effects model with weights in R

Specifically, I am trying to understand a random intercept model in the form of $${\bf y}={\bf X}{\boldsymbol \beta} + {\bf Z} {\bf b} + {\boldsymbol \varepsilon} $$

(Following Macro's notation for ${\boldsymbol y_i}$ in the previous question, I stack the group formulas together vertically and obtained one big formula for the whole dataset with all groups).

My questions are:

1. In a random intercept model (all group sharing the same beta), will the following situation arise: all data points in a group reside on one side (the same side) of the regression line of that group? Here is my thinking: even though for a certain group, the overall beta is a very crappy one for that group. By adjusting the intercept for that group, I at least still can get the regression line passing the cloud, am I right? Here by "passing the cloud" I mean not all points will reside on the same side of the regression line.

2. In a random intercept model, for each group, which point does the regression line pass thru? In the general OLS, we knew that it's the point $(\overline{x}, \overline{y})$ that is passed thru by the regression line. But for each group in a random intercept model, which point is that "central" point?

3. Moreover, in R, what's a convenient way to visualize what's happening within a certain group in a mixed model?

4. How do I understand the ${\bf b}$'s? What are these ${\bf b}$'s? May I say they are simply the $\overline{y}_i - \overline{y}$, i.e. the vertical axis of the group centers - the vertical axis of the whole-data center?

5. Why does lme explicitly impose that the sum of ${\bf b}$ is $0$ ?

6. To understand the lme models better, I am creating my own toy models and doing some comparative studies:

To cook my own model, I decide to solve the following least-square problem:

$$ \min \left( \sum_{k=1}^{K} w_{1k} {\bf b}_k^2 + \sum_{i=1}^{N} w_{2i} \varepsilon_i^2 \right) $$

here $\{ {\bf b}_k \}_{k=1}^{K}$, where $K$ is the number of groups. $\{ \varepsilon_i \}_{i=1}^{N}$, where $N$ is the number of individuals.

$w_1$ is a set of weights on the individuals and $w_2$ is a set of weights on the groups at the group level.

My purpose is to create a model that resembles the random intercept model, but ditching the Gaussian distribution assumption for the ${\bf b}$'s and I would like to have the regression line in each group(sharing the same beta) passing the data cloud for that group (Please ref. Q1 above).

• Do you think my model makes sense?

• Could you please help me critique my model vs. the lme random intercept model?

• Shall I add more constraints such as $\sum {\bf b}_k = 0$and $\sum \varepsilon_i =0$?

Thanks a lot for your help!

Following Macro's suggestion, I am creating a new thread for my questions. (It's somehow related to the original question,

which had been discussed here:

Understanding the linear mixed effects model equation and fitting a random effects model with weights in R

Specifically, I am trying to understand a random intercept model in the form of $${\bf y}={\bf X}{\boldsymbol \beta} + {\bf Z} {\bf b} + {\boldsymbol \varepsilon} $$

(Following Macro's notation for ${\boldsymbol y_i} in the previous question, I stack the group formulas together vertically and obtained one big formula for the whole dataset with all groups).

My questions are:

1. In a random intercept model (all group sharing the same beta), will the following situation arise: all data points in a group reside on one side (the same side) of the regression line of that group? Here is my thinking: even though for a certain group, the overall beta is a very crappy one for that group. By adjusting the intercept for that group, I at least still can get the regression line passing the cloud, am I right? Here by "passing the cloud" I mean not all points will reside on the same side of the regression line.

2. In a random intercept model, for each group, which point does the regression line pass thru? In the general OLS, we knew that it's the point $(\overline{x}, \overline{y})$ that is passed thru by the regression line. But for each group in a random intercept model, which point is that "central" point?

3. Moreover, in R, what's a convenient way to visualize what's happening within a certain group in a mixed model?

4. How do I understand the ${\bf b}$'s? What are these ${\bf b}$'s? May I say they are simply the $\overline{y}_i - \overline{y}$, i.e. the vertical axis of the group centers - the vertical axis of the whole-data center?

5. Why does lme explicitly impose that the sum of ${\bf b} is $0$ ?

6. To understand the lme models better, I am creating my own toy models and doing some comparative studies:

To cook my own model, I decide to solve the following least-square problem:

sum(w1kb_k^2)+sum(w2ieps_i^2)

here $\{ {\bf b}_k \}_{k=1}^{K}$, where $K$ is the number of groups. $\{ \varepsilon_i \}_{i=1}^{N}, where $N$ is the number of individuals.

$w_1$ is a set of weights on the individuals and $w_2$ is a set of weights on the groups at the group level.

My purpose is to create a model that resembles the random intercept model, but ditching the Gaussian distribution assumption for the ${\bf b}$'s and I would like to have the regression line in each group(sharing the same beta) passing the data cloud for that group (Please ref. Q1 above).

• Do you think my model makes sense?

• Could you please help me critique my model vs. the lme random intercept model?

• Shall I add more constraints such as $\sum {\bf b}_k = 0$and $\sum \varepsilon_i =0$?

Thanks a lot for your help!

Following Macro's suggestion, I am creating a new thread for my questions. (It's somehow related to the original question,

which had been discussed here:

Understanding the linear mixed effects model equation and fitting a random effects model with weights in R

Specifically, I am trying to understand a random intercept model in the form of $${\bf y}={\bf X}{\boldsymbol \beta} + {\bf Z} {\bf b} + {\boldsymbol \varepsilon} $$

(Following Macro's notation for ${\boldsymbol y_i} in the previous question, I stack the group formulas together vertically and obtained one big formula for the whole dataset with all groups).

My questions are:

1. In a random intercept model (all group sharing the same beta), will the following situation arise: all data points in a group reside on one side (the same side) of the regression line of that group? Here is my thinking: even though for a certain group, the overall beta is a very crappy one for that group. By adjusting the intercept for that group, I at least still can get the regression line passing the cloud, am I right? Here by "passing the cloud" I mean not all points will reside on the same side of the regression line.

2. In a random intercept model, for each group, which point does the regression line pass thru? In the general OLS, we knew that it's the point $(\overline{x}, \overline{y})$ that is passed thru by the regression line. But for each group in a random intercept model, which point is that "central" point?

3. Moreover, in R, what's a convenient way to visualize what's happening within a certain group in a mixed model?

4. How do I understand the ${\bf b}$'s? What are these ${\bf b}$'s? May I say they are simply the $\overline{y}_i - \overline{y}$, i.e. the vertical axis of the group centers - the vertical axis of the whole-data center?

5. Why does lme explicitly impose that the sum of ${\bf b} is $0$ ?

6. To understand the lme models better, I am creating my own toy models and doing some comparative studies:

To cook my own model, I decide to solve the following least-square problem:

$$ \min \left( \sum_{k=1}^{K} w_{1k} {\bf b}_k^2 + \sum_{i=1}^{N} w_{2i} \varepsilon_i^2 \right) $$

here $\{ {\bf b}_k \}_{k=1}^{K}$, where $K$ is the number of groups. $\{ \varepsilon_i \}_{i=1}^{N}, where $N$ is the number of individuals.

$w_1$ is a set of weights on the individuals and $w_2$ is a set of weights on the groups at the group level.

My purpose is to create a model that resembles the random intercept model, but ditching the Gaussian distribution assumption for the ${\bf b}$'s and I would like to have the regression line in each group(sharing the same beta) passing the data cloud for that group (Please ref. Q1 above).

• Do you think my model makes sense?

• Could you please help me critique my model vs. the lme random intercept model?

• Shall I add more constraints such as $\sum {\bf b}_k = 0$and $\sum \varepsilon_i =0$?

Thanks a lot for your help!