I have a sample of 20 subjects. I have two continuous variables, X and Y which are linearly related. I use linear regression to estimate the regression coefficient relating X and Y.

For each subject, I estimate a regression coefficient relating X to Y, and corresponding confidence intervals. So, I have a set of 20 regression coefficients and confidence intervals.

From this, how can I calculate a regression coefficient and confidence intervals for the entire sample?

Any suggestions much appreciated!

  • $\begingroup$ Your description is pretty vague, but you probably should use hierarchical regression in here, check e.g. stats.stackexchange.com/questions/134159/… $\endgroup$ – Tim Apr 4 '16 at 8:16
  • $\begingroup$ thank you, that's very helpful. it is the kind of thing I was looking for. $\endgroup$ – Nitin Apr 4 '16 at 8:45
  • $\begingroup$ Good. I provided a short answer with some references for you. $\endgroup$ – Tim Apr 4 '16 at 9:10

In cases as yours two extreme approaches can be taken: (a) calculate independent models for each of the individuals, or (b) calculate aggregate estimate for the whole sample, ignoring the individual variability. Unfortunately, both approaches can give you misleading results. When you calculate independent models you ignore the fact that they come from single population (so they may not be independent). When you calculate aggregated model, you ignore the individual differences. The patterns that can be observed on individual level are not the same as the ones that can be seen on global level, sometimes this is called atomistic and ecological fallacies.

If there is some kind of hierarchy or nesting, e.g. students nested within classes, classes within schools (see here for examples in bootstrap context), there is much wiser approach: hierarchical/multilevel regression. Such models simultaneously calculates effects for different levels (e.g. students, classes, schools). You can find example of such model described in here.

You can also check some of the multiple good and accessible handbooks on such models (check references below). There is also a friendly introductory paper by Bolker et al (2009) on (generalized) linear mixed models. Check also here to learn about the differences between random and fixed effects as knowing it may be helpful in the future.

Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J. S. S. (2009). Generalized linear mixed models: a practical guide for ecology and evolution. Trends in ecology & evolution, 24(3), 127-135.

Snijders, T.A.B. & Bosker, R.J. (2012). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. London: Sage Publishers.

Hox, J. (2010). Multilevel Analysis: Techniques and Applications. New York: Routledge.

Gelman, A. & Hill, J. (2006). Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press.

Pinheiro, J., & Bates, D. (2006). Mixed-effects models in S and S-PLUS. Springer.

  • $\begingroup$ wonderfully educational answer, thank you very much. $\endgroup$ – Nitin Apr 4 '16 at 9:16
  • $\begingroup$ can I use multilevel regression even there are differences in the number of cases across subjects? $\endgroup$ – Nitin Apr 4 '16 at 10:40
  • $\begingroup$ It is hard to answer without seeing your data and knowing the problem deeper, but the general answer is yes -- you'll be probably using random effects for this. $\endgroup$ – Tim Apr 4 '16 at 10:40
  • $\begingroup$ hi, one more quick question. I am using MATLAB function fitlme, to estimate fixed and random effects for a multi-level linear regression model. what about if I want to estimate a multi-level model for e.g. mutual information rather than linear regression? i.e. it should give me estimates of fixed and random effects for mutual information rather than linear regression... $\endgroup$ – Nitin Apr 12 '16 at 14:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.