Search Results

This should give an indication of the reliability of your measures, or in this case diagnostic agreement, at the two occasions. … your bonus question, you generally need 3 time points to separate the lack of (temporal) stability -- which can occur if the latent class or trait your are measuring evolve over time -- from the lack of reliability …

.† † E.g., assuming a theoretical reliability of 0.8, we want to determine the number of subjects needed to be 95% confident that the observed reliability estimated from a random sample is > 0.7; this …

You should use the point estimate of the reliability, not the lower bound or whatsoever. I guess by lb/up you mean the 95% CI for the ICC (I don't have SPSS, so I cannot check myself)? … It's unfortunate that we also talk of Cronbach's alpha as a "lower bound for reliability" since this might have confused you. …

My first idea would be to try some kind of cluster analysis (e.g. hierarchical clustering) on each similarity matrix, and compare the classification trees across raters. We can derive a similarity ind …

There's a nice summary of the use of Kappa and ICC indices for rater reliability in Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial, by Kevin A. … Usually, we use a two-way random effects model (both raters and items are treated as random effects) to estimate relative agreement (or the ICC(2,k) version for absolute reliability, if you care about …

Maybe I misunderstood the question, but what you are describing sounds like a test-retest reliability study on your Q scores. … Note that this reliability is sample-dependent (as it depends on the between-subject variance). …

You can compute an intraclass correlation coefficient, as described here Reliability in Elicitation Exercise. …

Such scenario are really parallel form of the same questionnaire, for which split-half reliability makes sense. …

My opinion is that you can go on with you ICC, but be sure to check that (a) there's a significant effect for the target of ratings (otherwise, it would mean the reliability of measurements is low), (b …

To sum up, correlated measurement errors, multidimensionality or unequal factor loadings make both indicators likely to diverge, with hierarchical $\omega_h$ being the reliability measure to use, following …

-- this has to do with scores reliability; Are the items behaving as expected for everybody, i.e. without differential effects depending on country or native language? …

although there are lot of debates about the optimal number of response options for Likert-type items (5 to 7 appear to be the most frequently solution), the problem is not so much how it affects the reliability …

Their simulation studies lead them to conclude that ordinal reliability alpha provides "consistently suitable estimates of the theoretical reliability, regardless of the magnitude of the theoretical reliability … Of note, there are alternative ways to estimate the reliability of test scores, see e.g., Zinbarg et al. (4). A good review is Bruce Thompson. Score Reliability. …

The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? (International Journal of Public Health 58(4):637-642). … Beware also that using a statistic like "alpha if item is deleted" without cross-validation is proned to over-fitting (the alpha is already a lower bound for reliability, but sample reliability may appear …

There is also a complete on-line tutorial on reliability analysis (Robert A. Yaffee) [archived version]. … For theoretical consideration about the mixed-effect approach, please consider reading my answer to this related question: Reliability in Elicitation Exercise. …