This is slightly embarrassing, as I've done a fair amount of statistical work, but for years I've heard this niggling voice at the back of my head, and I need to ask someone.
I remember when I first heard about multivariate regression and an example being used was a cancer trial of patients receiving surgery vs. chemotherapy (if I recall). Initially it appeared there was an advantage of one arm over another in terms of survival (days until death), but it turned out that there was a confounding covariate (e.g. age was much higher in one arm), and when this was factored in, there was no survival advantage for each surgery or chemotherapy.
When I've done covariate analysis, you obviously get a regression estimate for each variable, with the others taken into account.
I have a few questions based on the above examples
What would happen if you supplied two co-variates to the example above, the first being age in years, and the second being number of candles on your birthday cake.
Some people wont have any candles on their birthday cake, but lets say that generally people had more when they got older, and it was highly (but not quite perfectly) correlated.
Now candles aren't causative here, but would they falsely come out as having an adverse effect on mortality here? I know without age in the model, they would, but with age included, wouldn't they still? If not, that implies the model is able to ascertain that 'age' is to blame and 'candles' are not, which is obviously not possible for a statistical test. If so, do they 'share' some of the predictive power, and make 'age' look less important? Isn't this wrong? Is there some particular practice we should stick to to ensure we aren't including stuff like candles into models?
And a related example to the above, what would happen if we supplied not only the age in years, but a second parameter which was the age in days? If we supply more and more correlated columns, what happens?