I read a paper and it says "we regressed out the effect of a variable via linear regression".
Can someone provide to recipe to remove an effect of both numerical and categorical variables?
For example, remove the age effect: I know that my variable of interest depends on age and I want to remove that dependency.
Remove the site/measurement effect: I know that my variable of interest depends on the site of measurements -- on one site the variable is underestimated, on other it's overestimated, and I want to remove that dependency.
How I may act in the latter case -- I would split the whole data into the subpopulations depending on the sites, calculate overall mean of the whole data, and the means of the subpopulations and will add the difference between the overall mean and the subpopulation mean to the each subpopulation.
But how it's related to "regress out" and linear regression? For me a linear regression is just a line you can fit to the data, how one can relate the process of removing effect to that line and give a geometric intution? And how to deal with numerical data where I can't break the data into subpopulations?