What is covariate?

I'm confused with this term: covariate. What is it? Is is just the observed outcomes of some random variables that contain information that could help us enhance our prediction of another random variable that we haven't observed yet? Why is it named so?

Also there seems to be another: independent variable. Independent of what? Why is it named so?

• It's almost a synonym of independent variables. And there is a related concept: covariate shift. – Lerner Zhang May 23 '19 at 22:41
• Thanks. The independent variable also confuses me. Independent of what? I guess it's a random variable that is somehow independent of another? Which one? – caveman May 23 '19 at 22:55
• Quote: "An independent variable is the variable that is changed or controlled in a scientific experiment to test the effects on the dependent variable. A dependent variable is the variable being tested and measured in a scientific experiment." from this blog. – Lerner Zhang May 23 '19 at 22:57
• So "dependent" variable is clearly dependent on those other variables that are called "independent variables" in a cause-effect sense. But what are the "independent variables" independent from? Are they independent from each other? This is a part that confuses me: independent from what? Why is it named so? – caveman May 23 '19 at 23:03
• I think you are likely to find this post helpful. Some say the answer of what a covariate is depends on the context, so this gets a bit murky and confusing. See here for a pretty good explanation: theanalysisfactor.com/confusing-statistical-terms-5-covariate – StatsStudent May 24 '19 at 14:37

From Wikipedia:

Depending on the context, an independent variable is sometimes called a "predictor variable", regressor, covariate, "controlled variable", "manipulated variable", "explanatory variable", exposure variable (see reliability theory), "risk factor" (see medical statistics), "feature" (in machine learning and pattern recognition) or "input variable." In econometrics, the term "control variable" is usually used instead of "covariate".

• Assume that you are solving linear regression, where you are trying to find a relation $$\textbf{y} = f(\textbf{X})$$. In this case, $$\textbf{X}$$ are independent variables and $$\textbf{y}$$ is the dependent variable.
• Typically, $$\textbf{X}$$ consists of multiple variables which may have some relations between them, i.e. they "co-vary" -- hence the term "covariate".

Let's take a concrete example. Suppose you wish to predict the price of a house in a neighborhood, $$\textbf{y}$$ using the following "co-variates", $$\textbf{X}$$:

• Width, $$x_1$$
• Breadth, $$x_2$$
• Number of floors, $$x_3$$
• Area of the house, $$x_4$$
• Distance to downtown, $$x_5$$
• Distance to hospital, $$x_6$$

For a linear regression problem, $$\textbf{y} = f(\textbf{X})$$ the price of the house is dependent on all co-variates, i.e. $$\textbf{y}$$ is dependent on $$\textbf{X}$$. Do any of the co-variates depend on the price of the house? In other words, is $$\textbf{X}$$ dependent on $$\textbf{y}$$? The answer is NO. Hence, $$\textbf{X}$$ is the independent variable and $$\textbf{y}$$ is the dependent variable. This encapsulates a cause and effect relationship. If the independent variable changes, its effect is seen on the dependent variable.

Now, are all the co-variates independent of each other? The answer is NO! A better answer is, well it depends!

The area of the house ($$x_4$$) is dependent on the width ($$x_1$$), breadth ($$x_2$$) and the number of floors ($$x_3$$), whereas, distances to downtown ($$x_5$$) and hospital ($$x_6$$) are independent of the area of the house ($$x_4$$).

Hope that helps!

• Thanks, I read that. So I guess what I said is true? I.e. covariates are random variables that have their outcomes observed, and such observation is used to predict the outcome of unseen random variables of interest? – caveman May 23 '19 at 22:20
• Thanks! (re your own text below the Wiki quote). So, can I say that if $X$ are independent variables, then each such variable is giving me information such that all of the information is unique (no other variable gives me), but in case they have some relations (i.e. not perfectly independent --- aka co-varying), then each variable is giving me information that is partly novel, but also partly also given to me by other variables? In other words, if I compress the random variables by a perfect compression, can I say I will have independent random variables (no co-vary)? – caveman May 23 '19 at 23:00
• When we call X as independent variables, they are independent with regards to thier relation with y. X can consist of variables that can be independent or dependent on each other. Ideally you want all variables in X to be independent, which can be achieved using something like PCA. – kedarps May 23 '19 at 23:07
• THANKS THAT IS AMAZING. So I guess the name "independent" is from the perspective of cause-effect perspective, where we think that $X$ is what causes the price change, but me simply bumping the price of my house, it won't make my house suddenly become bigger? Is this the perspective from which "independent" is used? In other words, by "independent", they do not mean statistical independence as they are still statistically dependent. Am I right? Any mistakes? – caveman May 24 '19 at 14:05
• Note that independence doesn't imply a lack of a cause and effect relationship. See this post: stats.stackexchange.com/questions/357255/… – StatsStudent May 24 '19 at 14:34

the way linear regression is generally run (there are ways to ask for different slope calculations) you are getting the unique impact of one predictor on the dependent variable. Its shared impact with other predictors on the DV (or indirect impact as with structural equation models I believe) is not part of the slope. it is sometimes stated that the slope is the impact of a specific predictor setting all other X to zero (although that breaks down obviously when some X can not take on the value of 0 or you have interaction).

In general terms, covariates are characteristics of the participants in an experiment. If you collect data on characteristics before you run an experiment, you could use that data to see how your treatment affects different groups or populations. Or, you could use that data to control for the influence of any covariate.

Covariates may affect the outcome in a study. For example, you are running an experiment to see how corn plants tolerate drought. Level of drought is the actual “treatment,” but it isn’t the only factor that affects how plants perform: size is a known factor that affects tolerance levels so that you would run plant size as a covariate.

Another example (from Penn State): Let’s say you are comparing the salaries of men and women to see who earns more. One factor that you need to control for is that people tend to earn more the longer they are out of college. Years out of college, in this case, is a covariate.

A covariate can be an independent variable (i.e., of direct interest) or it can be an unwanted, confounding variable. Adding a covariate to a model can increase the accuracy of your results.