3
$\begingroup$

I have used software before to do linear regression and factor in/out the confounding variables, but what I would like to do is generate a new data set which is adjusted for the confounding variables. How would I go about this?

Further explanation:

I have 1 dependant variable, multiple interesting independant variables and multiple confounding independant variables.

I would like to

  1. Adjust the dependant variable for the confounding variables.
  2. Do PCA analysis on the interesting independant variables.
  3. Colour my data points on the PCA plot by the adjusted dependant variable.

I can do steps 2 and 3 but don't know how to do step 1. I can use R and have used MiniTab and SPSS once or twice, so any specific instructions relating to those would be amazing but even to be pointed in the right direction would be great.

$\endgroup$
4
$\begingroup$

In the context of multiple linear regression, "adjusting" for a covariate simply means including it as an explanatory variable.

There is an equivalent way of understanding multiple linear regression that provides insight into this question. To regress, say, $Z$ on $X$ and $Y$, we may (arbitrarily) select one of the explanatory variables (let it be $X$) and

  1. Separately regress $Z$ on $X$ and $Y$ on $X$, producing residuals $Z_X$ and $Y_X$, respectively, then

  2. Regress $Z_X$ on $Y_X$.

The first step "takes $X$ out of both $Y$ and $Z$." Its interpretation and legitimacy are the same as for any ordinary regression. The generalization to more than two explanatory variables should be clear.

After step 1, you do not have to proceed to step 2: you may study the relationships among the residuals in any ways you please, including PCA. The residuals, by construction, will be orthogonal to a constant (they will have zero means) and also orthogonal to the covariates that have been "left out" ("adjusted for").

You can adjust for interactions and nonlinear terms in the same way: create variables for them and include those in the variables that are taken out.


I will illustrate with R code.

set.seed(17)
#
# Create independent variables.
#
n.rows <- 30                    # Data count
n.vars <- 4                     # Number of explanatory variables
n.adj <- 2                      # Number of covariates to adjust for
x <- matrix(rnorm(n.rows * n.vars), nrow=n.rows)
#
# Create a dependent variable.
#
intercept <- -1
beta <- 1:n.vars                # Coefficients
sigma <- 0.5                    # "Error" SD
eps <- rnorm(n.rows, 0, sigma)  # "Errors"
y <- intercept + x %*% beta + eps 
#
# Multiple regression.
#
#pairs(data.frame(x,y))
summary(lm(y ~ x))
#
# Take out the first `n.adj` variables.
#
x.adj <- x[, 1:n.adj, drop=FALSE]     # Variables to take out
x.left <- x[, -(1:n.adj), drop=FALSE] # Variables that will be left
x.res <- apply(x.left, 2, function(y) residuals(lm(y ~ x.adj)))
y.res <- residuals(lm(y ~ x.adj))
#
# Compare the multiple regression of the residuals to the original regression.
#
#pairs(data.frame(x.res, y.res))
summary(lm(y.res ~ x.res))
#
# PCA of the residuals
#
pca <- princomp(cbind(x.res, y.res))
summary(pca)
biplot(pca)
# etc., etc.
| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I don't usually like to use very special cases that are in essence "tricks" to obtain the right answer. This trick does not work with nonlinear models and especially with semi-parametric models such as the proportional odds model. $\endgroup$ – Frank Harrell May 10 '13 at 19:36
  • 1
    $\begingroup$ @Frank Those are good points, although perhaps a little extreme (because "taking out" variables applies in some forms of nonlinear and robust regression, too). I believe the first line of the question indicates that your objections don't apply in the present situation, but it is good to know the limitations. I also would hesitate to characterize this solution as a "trick"; for instance, it is the conceptual and computational basis for most of Tukey & Mosteller's book on data analysis and regression. $\endgroup$ – whuber May 10 '13 at 19:46
  • 1
    $\begingroup$ Right, I used too strong a word. But I've never found this to be completely helpful - I'm thinking now of a polytomous independent variable. Because semiparametric regression models, and others, have many advantages over OLS, I don't prefer to use methods that are particular to OLS. $\endgroup$ – Frank Harrell May 11 '13 at 13:19
3
$\begingroup$

This project will take an excellent understanding of statistics. This is not really a software question.

Briefly, it is not useful to think of adjusting variables to produce new variables. You need to adjust for variables in the context of a statistical model. Only in the very special case of ordinary regression with no interactions and all effects linear (which is seldom the case) is it possible to even compute meaningful "adjusted variables."

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.