1
$\begingroup$

I got some survey data, in which the respondents answered various questions about environmental conditions on a 1-5 Likert scale. The respondents are also assigend to a household by an household ID. So e.g. there are 1 person living in household A, 3 persons in household B, 2 person in household C, 1 person in household D, etc.

Now I would like to check, whether the answers to the questions about environmental conditions are consistent within households. Or at least if it's more likely for persons living within a household to give the same (or a similar) rating on environmental conditions than people living in different households. (The hypothesis behind this is that people living in the same household should face similar conditions).

But I am kinda stuck on how to test that (in R)... any ideas/advice? :)

Thank you very much!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Two approaches that you can use:

  1. Compare the proportion of variance within an between groups - this is called the intra-class correlation (ICC). An intraclass correlation of 1 means that there is no variance within a group (all the members of the group agree) and there is variance between groups. An ICC of zero means that there is as much variance within groups as between groups - a person is no more likely to give a similar answer as someone in their household as they would be to someone outside their household.

  2. Use some flavor of Cohen's Kappa - which assesses the extent to which agreement is greater than would be expected by chance. (E.g. if everyone has a 90% chance of saying 'yes' then you expect 82% agreement by chance - 82% agreement would therefore be a kappa of 0, representing only chance agreement.

There are several packages for calculating each of these. Use your favorite search engine.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hey Jeremy, thank you very much for your answer! That's already helpful, but I still struggle to apply any of the functions in the various packages to my problem. I want to answer the following question: Is agreement on e.g. level of air pollution (rated on a one 1 to 5) higher within households than in the whole sample? Could you provide me with a small code example with any of the suitable packages (e.g. the ICC package)? I don't get where to specify the groups it should compare (household IDs) and which variable to test on (air pollution)... Thanks for your help! $\endgroup$
    – Felix D.
    Commented Jun 26, 2022 at 9:15
  • $\begingroup$ Well, I made some progress now. The samplesize4survey package (cran.r-project.org/package=samplesize4surveys) offers an ICC function that allowed me to do what I wanted (it almost seems to easy to do it with that package, haha). However, I also want to check for a binominal factor variable ("good" / "bad") whether responses within households are more consistant than in the whole sample population, which isn't possible using the ISS function. Any ideas on how to do that? $\endgroup$
    – Felix D.
    Commented Jun 26, 2022 at 10:15
  • $\begingroup$ Can you provide some example data? The binomial is trickier, but still possible. $\endgroup$
    – Jeremy Miles
    Commented Jun 26, 2022 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.