1
$\begingroup$

I am running a multi-level model on data which is set out with one row per individual survey return, each with Individual ID and Organisational ID as columns. There are variables which can be aggregated in to group, but I first have to justify this aggregation, and believe I have to use ICC(1), ICC(2) and Rwg. Although I can find a lot explaining what these are, I cannot seem to figure out how to do this with my output in SPSS.

I at first believed it came from the output of a one-way ANOVA, but have since been told it should be two-way for ICC(2) as this is a two-level multi-level model.

Can anyone help me on this one? I can't see how I can do a two-way ANOVA as I surely only need to use the Organisational ID and aggregating variable in question and two-way requires three variables?

Any help would be very gratefully received. Thanks!

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Care to elaborate on what ICC(1), ICC(2) and Rwg are? I know ICC stands for intra-class correlation, but what is the distinction between 1 & 2? Also I do not know what the acronym Rwg refers too. $\endgroup$
    – Andy W
    Commented Apr 18, 2012 at 17:04
  • $\begingroup$ Hi, apologies I should have elaborated further. The information I have is that ICC(1) is used when each subject is rated by multiple raters, raters assumed to be randomly assigned to subjects, all subjects have the same number of raters and ICC(2) is used when all subjects are rated by the same raters who are assumed to be a random subset of all possible raters. $\endgroup$
    – Sam Cruickshank
    Commented Apr 18, 2012 at 18:21
  • $\begingroup$ Rwg is a measure of interrater agreement within a single unit within a single measure, calculated by "comparing an observed group variance to an expected random variance" I have ICC(1) equation as (BMS) - (WMS)/(BMS + (k+1)WMS) and ICC(2) as (BMS-EMS)/(BMS+(k+1)EMS+k(JMS-EMS)/n where BMS is Between mean square, EMS is error, WMS is within mean square and JMS is between judges mean square. My data is individual survey data in rows where each is part of a group, signified by an organisation ID column. $\endgroup$
    – Sam Cruickshank
    Commented Apr 18, 2012 at 18:22
  • $\begingroup$ I can't figure out if only ICC(1) or both apply to my type of data and which variables must be included in an ANOVA to continue. Hope this clarifies matters. Ref: Chang, 2007 Testing a Theoretical Model for Severe Medication Errors and Shrout and Fleiss, 1979, Intraclass Correlations : Uses in AssessingRater Reliability $\endgroup$
    – Sam Cruickshank
    Commented Apr 18, 2012 at 18:22

1 Answer 1

Reset to default
2
$\begingroup$

I just came across this article

LeBreton, J. M. & Senter,J. L. (2008). Answers to 20 Questions About Interrater Reliability and Interrater Agreement. Organizational Research Methods,11, p 815-852.

I'm still working through it, but I think it will be of use.It provides SPSS syntax for rWG.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.