What exactly does it mean to "regress out" a variable or quantity? This is something I hear about all the time, but it's strangely difficult to find a precise description online. Any pointers - especially to algorithmic descriptions - would be much appreciated.
Although it is not the most commonly used phrase (in my experience), as used in the question you linked to, to "regress out" a third variable is synonymous with "partialling out," "controlling for," "adjusting for," or "while holding constant the value of" another predictor variable. These concepts have been explained at various places on this site; see for example the collection of links in this answer.