0
$\begingroup$

I ran a hierarchical logistic regression with 2 blocks/predictors:

I found out that block 1 is not a significant predictor, but block 2 is. Still: the odds ratio for block 1 is MUCH higher (2.90) than for block 2 (1.31) and increases when the second block is added. How can it be so high but still not significant? Could that be caused by an underlying effect? Or could that simply be, because my sample seize is too small, so the values are not reliable?

$\endgroup$
5
  • $\begingroup$ I'm unclear exactly what you did. Was it: Model 1 logit(Y)~ Block 1, model2: logit(Y) ~ Block 1 + Block 2? And what are the blocks? Single variables? Of what sort? $\endgroup$
    – Peter Flom
    Feb 9, 2014 at 11:22
  • $\begingroup$ @PeterFlom Exactly, that was the model. Each block contains one quantitative variable. variable 1 = standarized residuals of linear regression for each individual. those residuals resulted from predicting pre-test-values out of post-test-values (this was done to get a value for the individual learning potential), variable 2 = lobal score of a working skill assessmentstandarized residuals of linear regression for each individual. those residuals resulted from predicting pre-test-values out of post-test-values (this was done to get a value for the individual learning potential) $\endgroup$
    – Lara
    Feb 9, 2014 at 12:27
  • $\begingroup$ It is very unusual to use residuals from one model as variables in another. What are you trying to do? $\endgroup$
    – Peter Flom
    Feb 9, 2014 at 12:49
  • $\begingroup$ @PeterFlom I was trying to define the change score in a learning potential test from time 1 to time 2, based on Weingartz, Wiedl & Watzke (2008). There is says:"In contrast, and using statistical parameters of CTT, measures based on residuals of linear regression use the differences between the actual and predicted posttest score to reflect individual learning potential under consideration of the regression effect of the sample. Assuming that error values are perfectly random, the residuals can be taken as an estimate of the differential treatment effect." Or did I get that wrong? $\endgroup$
    – Lara
    Feb 9, 2014 at 13:00
  • $\begingroup$ When I talked about that with my professor he said I could use those residuals as a continous variable in the logistic regression. $\endgroup$
    – Lara
    Feb 9, 2014 at 13:01

1 Answer 1

0
$\begingroup$

I see several issues here:

  1. Ignoring that the independent variables are residuals, there are reasons that block 1 could be nonsignifcant despite having a high OR. One is that there is a lot of error (that would show up as the standard error in output from a program). And, any effect will become significant with a large enough N.

  2. Similarly, block 2 could be significant when block 1 is not, even if its OR is lower. This could happen because the standard error is lower. It could also be an effect of the fact that the model with both blocks controls for block 1. However, this cannot be due to sample size, as the sample is the same size for both models.

  3. The issue of using residuals as independent variables measuring "learning potential". This seems problematic, although I haven't read the article by W W and W. First, residuals can be negative. What would "negative learning potential" mean? Ability to forget? Second, residuals (in a situation like this) are composed of two things: The effect of variables that aren't in the model (I think this is what the authors are trying to get at) and noise. Why might a student get a score lower than his/her true ability? Maybe he didn't sleep well. Maybe he got in a fight with his mother that morning. Maybe his pencil slipped and he filled in the wrong circles. Who knows?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.