8
$\begingroup$

Please help me to interpret the findings of my model. The specifications of the model are:

Dependent variable: treatment (1) or no-treatment (0).

Independent variables: age, number of drugs used, comorbidity, others...

Multilevel structure: patients clustered within hospitals. Treatment rate varies across different hospitals. Multilevel logistic regression was used.

Findings: First, I ran the empty model with random intercept only and estimated the variance component (between hospital variance in treatment rate). Second, I added independent variables to the model one by one. Adding these variables either decreased or increased the variance component when comparing to empty model. Third, I added all the independent variables together into the model and variance component increased when comparing to empty model.

Conclusion: If I add variable to the model and variance decreases - this variable explains part of between-hospital variance in treatment rate. If I add variable to the model and it does not change variance component - this variable does not explain between-hospital variance.

Question: Please, give me an advice how can I interpret the fact that adding some variables to the model increase variance component.

Thank you in advance for any suggestions!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
9
$\begingroup$

Unfortunately, your conclusion is based on the logic of a linear multilevel model. For multilevel models with a binary outcome, the same logic does not hold. Please find below a brief explanation from Hox (2010: 133f):

"However, in logistic and probit regression (and many other generalized linear models), the underlying latent variable is rescaled, so the lowest-level residual variance is again $\pi^2/3$ or unity, respectively. Consequently, the values of the regression coefficients and higher-level variances are also rescaled, in addition to any real changes resulting from the changes in the model. These implicit scale changes make it impossible to compare regression coefficients across models, or to investigate how variance components change. Snijders and Bosker (1999) discuss this phenomenon briefly; a more detailed discussion is given by Fielding (2003, 2004)."

Hox, J. (2010). Multilevel Analysis: Techniques and Applications. New York: Routledge.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thank you for your comment, Bernd! That's what I suspected. This is sad, since I invested quite some time in conducting these analyses. I read the whole chapter devoted to 'Discrete dependent variables' in the book written by Snijders and Bosker, but they have not stated it so clearly. I'll try to find the book of Hox and read further on this issue. $\endgroup$
    – Gregory
    Commented Jan 26, 2015 at 10:23
  • 2
    $\begingroup$ @Gregory Why is this "sad"? What is the main purpose of your analyses? Are you explicitly interested in the variance components? $\endgroup$
    – Bernd Weiss
    Commented Jan 27, 2015 at 4:53
  • 1
    $\begingroup$ The purpose is (1) to measure between-hospital variance in treatment and (2) to identify patient characteristics that can explain this variance. Maybe there is another way to do it? In ML linear regression this approach is straightforward, but in ML logistic patient-level variance is constant and equal to 3.29. Snijders & Boskers in their book and Merlo et al. (2005, 2006) indicate that ICC and proportional change in variance still can be used to assess variance change between models, saying, however, that it depends from the distribution of independent variable across cluster variable. $\endgroup$
    – Gregory
    Commented Jan 27, 2015 at 13:51
  • 3
    $\begingroup$ Ad (2) should be doable using AIC and BIC, right? Ad (1), though, is tricky. Maybe, Goldstein's 'variance partition coefficient' approach works for you. He has written a few papers about it, here is one of them: Variance partitioning in multilevel logistic models that exhibit overdispersion $\endgroup$
    – Bernd Weiss
    Commented Jan 28, 2015 at 4:36
  • 1
    $\begingroup$ Thank you for the reference. I've read Goldstein paper and used the 'latent variable' method to calculate ICC as suggested by Goldstein, Merlo, Snijders & Bosker, and others. Following your advice I've calculated BIC for each model when new variable has been added. BIC value decreases when independent variable is significantly associated with dependent variable. The between hospital variance still increases, as I mentioned before! I could not understand that: -2loglikelihood and BIC show that model fit improves, but variance increases. How could that be? $\endgroup$
    – Gregory
    Commented Feb 3, 2015 at 14:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.