I have several small datasets of few (9 to 12) observations each, and I do have to treat these datasets separately. For each dataset, I want to test for the relative contribution of two continuous predictors on my response variable. More precisely, I am more interest in one predictor (lets call it $X1$), but I have have to take the effect of the second predictor ($X2$) into consideration to "retrieve" it from the relationship between $Y$ and $X1$.

$Y = intercept + aX1 + bX2 + e$ ($X1$ and $X2$ both continuous and non-collinear)

I know that testing the effect of two predictors with as few as 9 to 12 observations is tricky! Thus, I am using the R function 'summ' (after a classical lm())from the package jtools to get the mean + confidence intervals of each estimate (a and b) after standardization (R function 'standardize'); and get a general idea of the effect size of each predictor for each of my small datasets eventhough I get something non-significant (which is, expectedly, often the case).

Here are my questions:

  • Is that OK to do multiple linear regression to get the estimation (+CIs) of $a$ (the slope of $X1$), and only use this estimation in further analyses? In other terms: can I then consider I am actually reporting the proper, remaining effect of X1 on Y after accounting for the effect of $X2$?

  • Should I run a separate model first (i.e., $Y = bX2 + e$), get the residuals, and then (and only then) fit a model with residuals($Y.X2) = aX1 + e$ ? Is there any difference (I would intuitively say there is not!) between this and what I was describing formerly?

Thank you all for your answers! Still learning about secrets of statistics but hoping to get better at it very soon :-)

  • 3
    $\begingroup$ Unfortunately, multiple regression with only 9-12 observations will not be extremely helpful... Could you do it? Yes. Is it mathematically sound? Yes. Is it helpful in predicting future values or estimating the relationship? Most likely, no. $\endgroup$ – ERT Jul 30 '18 at 21:04
  • $\begingroup$ lm() in R is the standard (single and multiple) linear regression function, in case you are wondering about it. $\endgroup$ – ERT Jul 30 '18 at 21:07
  • $\begingroup$ Thanks you very much for your answers, ERT. I indeed used the lm() function to get estimates of the a and b slopes (I have added the info on my question). I know I have few observations, unfortunately I have no choice but dealing with it. Though, since I am reporting both estimates and CIs (even though these are wide), I was hoping to work with these estimates afterwards, tempering this with regard to effect sizes (i.e. interpreting results with high caution) but I was not sure to be right, in particular regarding the two questions above. Thank you again! $\endgroup$ – Chrys Jul 31 '18 at 6:41
  • $\begingroup$ I would suggest to try and use a bayesian regression approach. Especially in cases where you have only a few observations this approach gives you a better impression how "confident" you can be about your coefficients (credible interval) $a$ and $b$. In addition you can use your knowledge as a "scientists" to define priors about the outcomes of the coefficients. $\endgroup$ – burton030 Jul 31 '18 at 9:13
  • $\begingroup$ I don't see any reason in your narrative for testing the effect of $X_2$: you need to control for it, so do so. That leaves you with 6 to 9 degrees of freedom to test $X_1$ in each case, which could suffice. To the extent your models might have a common value of some parameter (for instance, the variance of the errors, or perhaps even the intercept or the $X_2$ coefficient), you could "borrow strength" by fitting them all at once. One might view this as a classical version of the Bayesian approach suggested by @burton. $\endgroup$ – whuber Jul 31 '18 at 11:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.