Timeline for Can correlated random effects "steal" the variability (and the significance) from the regression coefficient?
Current License: CC BY-SA 3.0
19 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 9, 2014 at 18:26 | comment | added | Lars Lau Raket | @Curious. The model I write is a generic model, not related at all to your model. It is just a very simple example. Every $y_i$ in the model is an observation of $\alpha$ plus noise $x_i+\varepsilon_i$ and both terms are unknown. The reason I split the noise term in two is that you often have independent measurement noise $\varepsilon_i$ that is caused by errors in the measurement process, and some systematic noise caused by the system you are observing. The point is that these types of systematic noise (corresponding to $\gamma_j$ in your model) will have a specific effect on the likelihood. | |
| Oct 9, 2014 at 9:16 | history | bounty ended | Tomas | ||
| Oct 9, 2014 at 9:14 | comment | added | Tomas | Dear Lars, sorry for period of silence. I will award the bounty to you, though your answer is not clear to me yet. Could you please rewrite indices so that at least $x_i$ and $\epsilon_i$ doesn't use the same index $i$ when in fact they are indexed differently? I am still not able to get it even after your comment. You can use the same indices as I used in my answer. | |
| Oct 3, 2014 at 9:34 | comment | added | Lars Lau Raket | @Curious: Then there should be no problem computing $\hat\beta x_{i,j}$ where $\hat\beta$ is your estimate for $\beta$. You can compute all the values, and judge whether a 0.17 standard deviation seems to be a big deal around the values. Since you say $x_{i,j}$ have approximately zero mean, the 0.03 standard deviation does seem to imply that the contribution of $\beta$ in your model is small, but it may still be significant. | |
| Oct 3, 2014 at 9:28 | comment | added | Lars Lau Raket | The model I write is just the simplest example I could come up with. There are $n$ noisy observations. They contain iid noise in the form of $\varepsilon_i$ and noise $x_i$ that is correlated between the different observations (which is the most basic non-trivial example of a random effect). When I write $(y_i)_i$ I mean the $n$-vector consisting of all observations. When I write the distributions of the vectors, I write the covariance matrices $S$ and $\sigma^2 \mathbb{I}_n$ that describe the correlation between the variables. | |
| Oct 3, 2014 at 9:21 | comment | added | Tomas | Lars, exactly, $x_{i,j}$ are known environmental conditions, this is the independent predictor variable | |
| Oct 3, 2014 at 9:20 | comment | added | Lars Lau Raket | @Curious:$x_{i,j}$ is your environmental conditions? Don't you know them? | |
| Oct 3, 2014 at 9:18 | comment | added | Tomas | I don't understand your model and the notation. Are you sure the indices are ok? If $i$ is the observation, where is the index for the group the random effect applies to? $\epsilon_i$ is the residual then, not the random effect, but what is the $x_i$? Why does it have the same index as $\epsilon_i$? What does the $(y_i)_i$ notation mean? Please make sure the indices are precise because otherwise I have troubles understanding it. | |
| Oct 3, 2014 at 9:13 | comment | added | Tomas | Lars, but the values of $x_{i,j}$ have approx normal distribution with the mean = zero, so what is the "actual value of $\beta_i x_{i,j}$" then? Mean of this term is actually zero. | |
| Oct 3, 2014 at 7:59 | history | edited | Lars Lau Raket | CC BY-SA 3.0 |
deleted 263 characters in body
|
| Oct 3, 2014 at 7:57 | comment | added | Lars Lau Raket | @Curious: I have updated the answer. You should compare $\sigma$ to the actual values of $\beta_i x_{i,j}$ and not their variability. If a typical value of $\beta_i x_{i,j}$ is 0.00001 and the estimated standard deviation of your random effect is 0.17, it seems that $\beta$ does not explain a lot of variation. | |
| Oct 3, 2014 at 7:50 | history | edited | Lars Lau Raket | CC BY-SA 3.0 |
Expanded answer
|
| S Oct 2, 2014 at 14:56 | history | suggested | Tomas | CC BY-SA 3.0 |
Replacing $\gamma_i$ with $\gamma_j$ as it is in my question
|
| Oct 2, 2014 at 14:29 | review | Suggested edits | |||
| S Oct 2, 2014 at 14:56 | |||||
| Oct 2, 2014 at 14:27 | comment | added | Tomas | I have replaced $\gamma_i$ with $\gamma_j$ in your answer - I guess this is what you meant. | |
| Oct 2, 2014 at 14:25 | comment | added | Tomas | Note to point 2) above - I am not comparing "variance of $\sigma^2$" as you suggested but instead I am comparing $sd(\beta_ix_{i,j}) = 0.03$ with $\sigma$. I hope this is what you meant? | |
| Oct 2, 2014 at 14:21 | comment | added | Tomas |
Lars, thank you for your answer! I have a few questions: 1) why is it "cheaper to put the variability in $\beta$?" 2) Regarding the size of $\sigma^2$ vs $\beta_ix_{i,j}$, how do I compare it properly? Unfortunatelly there is a convergence problem and I have only one model run where estimate of $\sigma$ converged. The estimate is $\sigma = 0.17$, $\beta = -0.47$, $sd(x_{i,j}) = 0.06$ (should I compute sd of $x$? mean is zero), so this gives me $sd(\beta_ix_{i,j}) = 0.03$ which is much less than the $\sigma$! Does it mean that the variability is then consumed by the random effect?
|
|
| Oct 1, 2014 at 12:20 | review | First posts | |||
| Oct 1, 2014 at 12:25 | |||||
| Oct 1, 2014 at 12:17 | history | answered | Lars Lau Raket | CC BY-SA 3.0 |