Return to Answer

I think I can answer your questions at least concerning the unadjusted repeatability estimates, i.e., the classical intra-class correlations (ICCs). As for the "adjusted" repeatability estimates, I skimmed over the paper you linked and didn't really see where the formula that you apply can be found in the paper? It is also not immediately obvious to me what the adjustment is doing, so I will have to ignore the adjusted part of your question. It's not clear that this is a critical part of your question anyway. But if you think that it is, perhaps you can provide some clarification about the adjustment.

I think I can answer your questions at least concerning the unadjusted repeatability estimates, i.e., the classical intra-class correlations (ICCs). As for the "adjusted" repeatability estimates, I skimmed over the paper you linked and didn't really see where the formula that you apply can be found in the paper? Based on the mathematical expression, it appears to be the repeatability of mean scores (rather than individual scores). But it's not clear that this is a critical part of your question anyway, so I will ignore it.

I think I can answer your questions at least concerning the unadjusted repeatability estimates, i.e., the classical intra-class correlations (ICCs). As for the "adjusted" repeatability estimates, I skimmed over the paper you linked (didn't read closely) and didn't really see where the formula that you apply can be found in the paper. It is also not immediately obvious to me what the adjustment is doing, so I will have to ignore the adjusted part of your question. It's not clear that this is a critical part of your question anyway. But if you think that it is, perhaps you can provide some clarification about the adjustment.

Plugging these into the correlation formula gives us $$ ICC = \frac{cov(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1}, 2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1}}{\sqrt{var(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1})var(2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1})}}, $$ which simplifies down to $$ ICC = \frac{2x^2\sigma^2_{u_1}}{2x^2\sigma^2_{u_1} + \sigma^2_e}. $$ Notice that the ICC is technically a function of $x$! However, in this case $x$ can only take 2 possible values, and the ICC is identical at both of these values.

I think I can answer your questions at least concerning the unadjusted repeatability estimates, i.e., the classical intra-class correlations (ICCs). As for the "adjusted" repeatability estimates, I skimmed over the paper you linked and didn't really see where the formula that you apply can be found in the paper? It is also not immediately obvious to me what the adjustment is doing, so I will have to ignore the adjusted part of your question. It's not clear that this is a critical part of your question anyway. But if you think that it is, perhaps you can provide some clarification about the adjustment.

Plugging these into the correlation formula gives us $$ ICC = \frac{cov(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1}, 2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1})}{\sqrt{var(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1})var(2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1})}}, $$ which simplifies down to $$ ICC = \frac{2x^2\sigma^2_{u_1}}{2x^2\sigma^2_{u_1} + \sigma^2_e}. $$ Notice that the ICC is technically a function of $x$! However, in this case $x$ can only take 2 possible values, and the ICC is identical at both of these values.

The expression for ICC / repeatability coefficient then comes from letting the two random variables $x$ and $y$ be two observations drawn from the same $j$ group, $$ ICC = \frac{cov(\beta_0 + u_{0j} + e_{i_1j}, \beta_0 + u_{0j} + e_{i_2j})}{\sqrt{var(\beta_0 + u_{0j} + e_{i_1j})var(\beta_0 + u_{0j} + e_{i_2j})}}, $$ and if you simplify this using the definitions given above and the properties of variances/covariances (a process which I will not show here, unless you or others would prefer that I did), you end up with $$ ICC = \frac{\sigma^2_{u_0}}{\sigma^2_{u_0} + \sigma^2_e}. $$ What this means is that the ICC or "unadjusted repeatability coefficient" in this case has a simple interpretation as the expected correlation between a pair observations from the same cluster (net of the fixed effects, which in this case is just the grand mean). The fact that the ICC is also interpretable as a proportion of variance in this case is incidental; that interpretation is not true in general for more complicated ICCs. The interpretation as some sort of correlation is what is primary.

So what is the "repeatability of an effect" under this model? I think a good candidate definition is that it is the expected correlation between two pairs of $x_2-x_1$ difference scores computed within same $j$ cluster, but with different pairs of observations $i$.

Plugging these into the correlation formula gives us $$ ICC = \frac{cov(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1}, 2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1}}{\sqrt{var(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1})var(2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1})}}, $$ which simplifies down to $$ ICC = \frac{2x^2\sigma^2_{u_1}}{2x^2\sigma^2_{u_1} + \sigma^2_e}. $$ Note that the ICC is a function of $x$! However, in this case $x$ can only take 2 possible values, and the ICC is the same at both of these values.

If the other predictors are orthogonal to the predictor of interest (as in, e.g., a balanced experiment), I would think the ICC / repeatability coefficient elaborated above should work without any modification. If they are not orthogonal you would need to modify the formula to take account of this, which could get complicated, but hopefully my answer has given some hints about what that might look like.

The expression for ICC / repeatability coefficient then comes from letting the two random variables $x$ and $y$ be two observations drawn from the same $j$ group, $$ ICC = \frac{cov(\beta_0 + u_{0j} + e_{i_1j}, \beta_0 + u_{0j} + e_{i_2j})}{\sqrt{var(\beta_0 + u_{0j} + e_{i_1j})var(\beta_0 + u_{0j} + e_{i_2j})}}, $$ and if you simplify this using the definitions given above and the properties of variances/covariances (a process which I will not show here, unless you or others would prefer that I did), you end up with $$ ICC = \frac{\sigma^2_{u_0}}{\sigma^2_{u_0} + \sigma^2_e}. $$ What this means is that the ICC or "unadjusted repeatability coefficient" in this case has a simple interpretation as the expected correlation between a pair observations from the same cluster (net of the fixed effects, which in this case is just the grand mean). The fact that the ICC is also interpretable as a proportion of variance in this case is coincidental; that interpretation is not true in general for more complicated ICCs. The interpretation as some sort of correlation is what is primary.

So what is the "repeatability of an effect" under this model? I think a good candidate definition is that it is the expected correlation between two pairs of difference scores computed within the same $j$ cluster, but across different pairs of observations $i$.

Plugging these into the correlation formula gives us $$ ICC = \frac{cov(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1}, 2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1}}{\sqrt{var(2x\beta_1+2xu_{1j}+e_{i_1jk_2}-e_{i_1jk_1})var(2x\beta_1+2xu_{1j}+e_{i_2jk_2}-e_{i_2jk_1})}}, $$ which simplifies down to $$ ICC = \frac{2x^2\sigma^2_{u_1}}{2x^2\sigma^2_{u_1} + \sigma^2_e}. $$ Notice that the ICC is technically a function of $x$! However, in this case $x$ can only take 2 possible values, and the ICC is identical at both of these values.

If the other predictors are orthogonal to the predictor of interest (as in, e.g., a balanced experiment), I would think the ICC / repeatability coefficient elaborated above should work without any modification. If they are not orthogonal then you would need to modify the formula to take account of this, which could get complicated, but hopefully my answer has given some hints about what that might look like.