Regress IV on DV, or DV on IV?

Question

This is a question about statistical language. Do you regress the IV on the DV, or do you regress the DV on the IV? Which is the correct way of saying this?

Answer 1

Traditionally speaking, one regresses the dependent variable (the Y, the outcome) on the independent variable (the X, the input). However, this is such an egregious abuse of statistical language, many disciplines have abandoned such verbiage altogether. The mistake is that "dependence" (in the proper statistical sense) is commutative. If A depends on B, then B depends on A. We only call the "X" (input variable) "independent" because it is considered fixed or given as part of an experimental design, or is representative of a population of interest.

Regression models estimate the conditional mean of the outcome as a function of one or more predictors. To this end, the mean of one variable conditional on another may be a flat response although the variables are indeed dependent (suppose the error of the Y varies according to X). To belabor this point, one nice way of writing what a regression model estimates is the following:

$$E[Y|X] = \beta_0 + \beta_1 X$$

Better options would be calling the "dependent" variable (Y): an outcome, a response, an output, and calling the "independent" variable (X): an input, a predictor, a regressor, a covariate, or an exposure.

Answer 2

The question prejudges another question, good terminology for the variables concerned. Let's take that first.

DV is common, but not universal, shorthand for dependent variable. It's probably old-fashioned to remind that DV has often been used to mean Deo volente, God willing, but those who know that and also some statistics seem unlikely to confuse or conflate those two meanings.

IV is common, but not universal, shorthand for independent variable. It's not at all old-fashioned to point out that among many economists, and some other social scientists, IV is now more likely to mean instrumental variable. This isn't much of a problem either: by the time people have learned about instrumental variables, they should be able to distinguish the two usages, at least in context.

Let's take it for the moment that at least in many situations, a dependent variable can be identified on substantive grounds as whatever is the outcome, response or effect which we are in practice interested in explaining or predicting in some way. The independent variable is then the cause or factor used to predict the response. Most introductory courses and texts seem to use notation $y$ for dependent variable and $x$ for independent variable; whenever there are many such independent variables, they can be distinguished by subscripts and/or denoted collectively as a matrix. That said, there are many examples in which predictive interest runs either way: if rainfall is a predictor of the response corn yield, so also we might use abundance of some taxon in reverse to predict temperature, rainfall or salinity of past environments. Yet again, and very much to the point, there are many problems in regression in which variables are on the same footing: properties of partners or siblings, two methods of measurement of ostensibly the same property, rainfall or temperature over time at two gauges or stations, and so on and so forth. The problem here has more symmetry and distinction between two kinds of variables is likely to be arbitrary if not meaningless.

As far as terminology is concerned, we note that many would prefer some other term rather than DV or dependent variable. This preference goes back at least some decades: John Wilder Tukey often used the term response in writings in the 1960s and 1970s. but teachers, writers and researchers often seem reluctant to abandon the terminology of dependent and independent. Grounds for objection include (a) many students and even researchers confuse the two words, which apparently seem so similar; (b) the words have other meanings, even in probability and statistics; (c) why use dull words when evocative alternatives are available?

Similarly, many find terms such as predictor, covariate, explanatory variable more congenial for independent variable. There are many such terms, and some, particularly the first two, have other meanings in statistical science. (For example, covariate had a very specific meaning in analysis of covariance for some decades, but somehow has morphed into also acquiring a more general meaning as any kind of predictor. I conjecture that writings of John Nelder had some influence there.) Yet again, some have favoured terms invented for the purpose, such as regressand and regressor: to me, these are so unattractive that it is slightly distressing even to think about them.

All this is lengthy preamble to the question given here (which to me is less interesting). In short, the usual or standard regression is that of $y$ on $x$ (or $X$), but at least in the case of single predictors it can also make sense to talk about the regression of $x$ on $y$, with different assumptions on error structure. There can be equal interest in both regressions when variables are on the same footing (and also in other models for the joint relationship, which are left to another story).