0
$\begingroup$

I am performing a meta-analysis comparing two methods for performing a test; the traditional (TRAD) method and a new experimental (EXP) method. As a secondary analysis I want to look at the maximum measure obtained from a 0–10 likert scale, let's call it "Rating A". In TRAD, rating A is obtained from the participant by having them provide the rating at the end of the test. In EXP, the rating scale is used to guide the test which terminates when the scale reaches 10, hence the maximum value for EXP will ALWAYS be 10 with no variance.

Ordinarily, within a primary study, this would be assessed via a one-tail, one-sample t-test for whether the sample mean from TRAD is less than 10.

Is there an equivalent method for a meta-analysis where I can take the mean and variance in Rating A for TRAD and compare it to a defined value (i.e., 10)? Essentially, I want to test if the mean value across studies for Rating A from TRAD is less than 10, the maximum of the scale.

That said, I'm not convinced this is an appropriate analysis given the scale has a ceiling. Are there any reasons this would not be appropriate, or is there an alternative way to approach this analysis?

The primary analyses were performed using R and metafor.

$\endgroup$
3
  • $\begingroup$ Does the prediction interval from your meta-analysis of TRAD include 10? $\endgroup$
    – mdewey
    Commented Sep 9 at 16:19
  • $\begingroup$ If rating A comes from a 0-10 Likert scale, it is discrete, ordinal-scale data. A t-test is valid for continuous, interval-scale data. Your data is bounded (0-10). A t-test assumes that the sampling distribution of the mean is Gaussian (i.e. unbounded). Any one of theses issues, by itself, is not a show-stopper. But 3 strikes? $\endgroup$
    – jginestet
    Commented Sep 9 at 17:29
  • $\begingroup$ If I understand correctly, you want to look at several TRAD studies, use the reported means of rating A, and compute a CI for the mean of these means, and show that the CI does not include 10? Given that 10 is the max possible value, for the true mean of the means (which probably you should not compute; see earlier comment) to be 10, it would imply that all studies had a true mean of 10, which would imply that all subjects in all studies, had a 10... Is there any study with even a single subject not rating A as 10? QED; the true mean of the TRAD studies can not be 10... $\endgroup$
    – jginestet
    Commented Sep 9 at 17:34

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.