1
$\begingroup$

I am comparing the assessments of cognitive domains in 2 conditions: either in a noisy environment or in a no noisy environment. The same people are in both conditions. My next step is to see if there is a significant difference on the cognitive domain between the 2 conditions with a paired sample t-test and my hypothesis is that in the noisy condition people perform less then in the no noise. If the paired sample t test is significant i go to my regression and i calculate the change score (noise scores - no noise scores) and use this as the dependent variable. Here comes the problem ; some people show a negative change instead of a positive one so their change is in the opposite direction. I'm wondering how do I deal with this in my regression? Should I adjust it to absolute differences? I assume not because then you are saying people got better while they actually didn't?

$\endgroup$
2
  • 3
    $\begingroup$ I wonder if, in your design, you actually permute the condition so some people permute their testing experience as noise -> no noise whereas others are no noise -> noise sequentially. If not, there is a risk of learning effect - any test can be learned, even cognitive tests. $\endgroup$
    – AdamO
    Commented Aug 16, 2023 at 16:51
  • $\begingroup$ Hi yes we made sure people were randomally assigned to a different order either noise-no noise // no noise-noise $\endgroup$
    – Rosy
    Commented Aug 16, 2023 at 20:01

3 Answers 3

2
$\begingroup$

You certainly should not take absolute values of change.

In general, a regression can have a DV that is negative or positive or a mix of the two, that's not a problem.

However, change scores can be a problem, especially when the dependent variable is not very reliable (and no measure of cognition is very reliable). In essence, you can wind up regressing on the error (or noise).

See Multilevel model with two timepoints?

and

Linear regression or mixed effects models for data with two time points?

for some useful discussion and ideas.

$\endgroup$
3
  • $\begingroup$ thankyou for your help! How would you suggest i run my analysis and do you maybe have some articles that write about this? In my regression i am using the following predictors (Age, sex, cognitive complaints and personality traits) and then the change score noise-no noise as the predictor. i do not really understand what seems to be wrong with change scores and how i could fix this then? $\endgroup$
    – Rosy
    Commented Aug 16, 2023 at 19:57
  • $\begingroup$ The score a person gets on a test is partly true score (T) and partly error (E). When you use change scores, you are often regressing on the difference in E. Suppose Joe and Rahul are just as smart as each other. On day 1, Joe has a great day. He slept well, ate breakfast, came to school and took the test. But Rahul had a horrible time. His parents were fighting, his little brother woke him up in the middle of the night, he overslept and missed breakfast. $T_J = T_R$ but $E_J > E_R$. On day 2, the opposite happens. It looks like Rahul got a lot smarter and Joe a lot dumber. $\endgroup$
    – Peter Flom
    Commented Aug 16, 2023 at 21:07
  • $\begingroup$ Some people have even said you shouldn't look at change in this kind of situation. Multilevel models deal with this very well, but may require more time points. $\endgroup$
    – Peter Flom
    Commented Aug 16, 2023 at 21:08
3
$\begingroup$

The paired $t$-test is equivalent to a linear regression model blocking on subject or a mixed effects model with subjects as random effects. Examples here.

$\endgroup$
2
$\begingroup$

This is the case where the statistician should endeavor to be agnostic about their results.

The relations that we model with regression are not well ordered, they are stochastically ordered. It may be the case for a particular individual or setting that, even when receiving the superior condition or treatment, their outcome is worse than if they had not received it. Regression quite literally averages up over everything - the patient level characteristics, the settings, the experimental contamination, and so on. Certain forms of biases can be excluded, but outcomes contrary to expectation does not a bias make.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.