ANCOVA with correlated predictors

Question

I am doing comparisons between two groups (control and treatment) and would like to keep this question limited to the case of two groups, not the full-blown $k$-sample problem.

In addition to knowing the group memberships of my observations (100mL comes from control, 116mL comes from treatment, etc), I also know some covariates. My first thought was to put my data into a linear regression and test the group membership variable for significance. However, I believe or at least fear that the covariates are correlated. Therefore, the parameter inference will be dubious. I either won't be testing what I want to test, or I will lose power in my test.

How have others approached this issue and found success? I see plenty about how to compare fits between a regression with and without the group membership variable, but it all seems to assume a lack of correlation among the predictor variables in the regression. Or can I just F-test the model with the group membership variable (plus covariates) against the model with just the covariates and deduce that the group membership is significant if the former model has the better fit?

Limit the situation to OLS linear regression, so a standard ANCOVA for just one covariate. (I eventually want to do this for GLMs, too, but baby steps...)

Answer 1

(This self-answer assumes OLS linear regression, which is what I meant when I posted the question.)

It doesn't matter!

The concern I had four years ago was that, if there are correlated features, that can inflate the variance of the coefficient estimates. This happens through the variance-inflation factor, which considers the $R^2$ of a dummy regression that predicts a feature using all other features.

If that feature is unrelated to the other features, that $R^2$ will be zero (or close to zero, empirically). That is, the correlation between the covariates does not impact the variance of the estimation of my variable of interest (the indicator for control vs treatment), so long as the covariates are unrelated to the variable of interest. (Knowing what I was doing when I posted the question, such an assumption is at least a bit dubious, but that is a separate issue.)