For a reliability study I’m administering patient reported outcomes (questionnaires) with 2 modes of administration (on paper and via an electronic device). I’d like to measure equivalence/reliability of the 2 modes of administration. In other words: does the electronic version of the questionnaire yield comparable results as the paper version (for an individual patient)?

Study set-up:
 Patients are randomised in either of 2 groups: (1) a group that fills in the paper based version first and (2) a group that fills in the electronic version first. Both groups complete both modes of administration. There are 10 patients in both groups (total n=20).

About the questionnaires: Patients completed 3 questionnaires in both modes of administration. Each question in the questionnaires were scored on a 1-10, 0-3 or 1-5 scale. Total scores and subscale scores (if present) were calculated.

Question: My intuition tells me to use the ICC to measure equivalence. However, the data is not always normally distributed (performed a KS test of the difference between the 2 scores since the data is paired). Because of this, it seems wrong to use ICC as a statistical method to measure equivalence. Which methods (parametric vs. non-parametric) can I use to correctly assess equivalence/reliability between the 2 modes of administration (paper vs. electronic questionnaire)? Keep in mind I only have 10 patients per group and not all variables/scores are distributed normally.

I’m using SPSS for the statistical analysis.

Kind regards, Nick


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.