2
$\begingroup$

I am reading a publication in which the author mentioned that they used the paired t-test. In their study, a questionnaire was provided to a group of people before and after surgery to check the efficiency of the treatment. The questionnaire includes a very important question about the level of pain they feel (from 1 to 10). Some people did not respond to this question before or after surgery, resulting in unequal sample sizes. The article provides the information of mean and SD before and after surgery, (mean1, sd1, Mean2, sd2, along with the p-value (by applying the paired t-test or Wilcoxon signed-rank test).

My question: If the mean values they provided are based on the number of observations (n1: number of observations before surgery, n2: number of observations after surgery) and not on the paired observations (after removing the subjects who have missing information for either before or after surgery), did they present the information correctly?

Improve this question
$\endgroup$
2
  • $\begingroup$ You write "applying the paired t-test or Wilcoxon signed-rank test". Is this "or" because the authors did not specify which test they used?This is concerning because t-test and WSRt test 2 completely different nulls; and one can easily create data where the conclusions of these 2 tests would be opposite. If the authors did not specify the test (and you are just deducing), then this is a poor description of the methods. Moreover, the pain scores would be from Likert-type data, i.e. ordinal. Subtracting such scores is, at best, a questionable practice (albeit (too) commonly used). $\endgroup$
    – jginestet
    Commented Nov 18 at 17:44
  • $\begingroup$ It's really unclear form you question as to what is going on. Show us the data and then someone can make an assessment. $\endgroup$
    – stweb
    Commented Nov 19 at 10:57

2 Answers 2

Reset to default
0
$\begingroup$

I don't necessarily think reporting the sample sizes or descriptives in this way is bad in and of itself. As an example, suppose there is a substantial difference in Time 1 and Time 2 complete cases. Knowing what the observed sample means/SDs were before their missingness treatment would give some sense of potential biasing effects if the inferential tests come out wildly different from expectation (e.g. poor imputation or just tons of data thrown out).

The more serious problems are if 1) authors are not reporting the inferential estimates and the actual final sample sizes for their t-tests or 2) information about missingness treatment is being omitted from the article. If that's the case, it would be disingenuous to leave out that information, particularly with how much is known about missingness biasing statistical tests.

Improve this answer
$\endgroup$
0
$\begingroup$

It's potentially misleading depending on how it's presented - I would say reporting multiple statistical parameters within the same set of parenthesis alongside a p-value suggests a single statistical test of those parameters, and can be misleading if it in fact represents multiple distinct tests.

It's not incorrect to run an unpaired test and report the unpaired sample size and p-value, and it's not incorrect to run a paired test and report the paired sample size and p-value. It would be incorrect to suggest the p-value came from a different statistical test, though, so one shouldn't present the paired p-value directly alongside the unpaired N - I would avoid putting them in the same set of parenthesis, as that usually describes the inputs/outputs of a single statistical test. The t-test is characterized by the sample sizes, means, SDs, t-statistic, degrees of freedom, and p-value - we shouldn't mix values corresponding to different tests and report the p-value from one test alongside the N from a different one. You would never, for example, report a chi-squared N, DoF, and statistic, and the p-value from a completely different statistical test on a different set of individuals.

It sounds like the publication was presented clearly enough that you understood the paired p-value did not correspond to the unpaired N, although it's not clear if that was explicit in the publication or if it required some "detective work" on your part. Running the test wasn't wrong, but the presentation may or may not be "correct" depending on if it is clear or obfuscates the methodology.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.