# What is covariate?

I'm confused with this term: covariate. What is it? Is is just the observed outcomes of some random variables that contain information that could help us enhance our prediction of another random variable that we haven't observed yet? Why is it named so?

Also there seems to be another: independent variable. Independent of what? Why is it named so?

• It's almost a synonym of independent variables. And there is a related concept: covariate shift. Commented May 23, 2019 at 22:41
• If features $X$ are independent from the dependent variables, then I guess it means that $\Pr(Y|X) = \Pr(Y)$ which makes the features useless. So I think your guess is wrong. Commented May 24, 2019 at 1:41
• If features X are independent from the dependent variables, then I would guess it means that $Pr(X|Y)=Pr(X)$ Commented May 24, 2019 at 1:46
• It means both. I.e. $\Pr(X|Y) = \Pr(X)$, and $\Pr(Y|X)=\Pr(Y)$. So, $\Pr(X,Y) = \Pr(X)\Pr(Y)$. And since $\Pr(X)$ is constant for a given problem, e.g. $\Pr(X=x_1)\Pr(Y=y_1)$ and $\Pr(X=x_1)\Pr(Y=y_2)$, you can drop $\Pr(X=x_1)$ altogether. Which means you will end up classifying only based on $\Pr(Y)$, which is fairly useless, specially if you have balanced classes. Commented May 24, 2019 at 2:00
• I think you are likely to find this post helpful. Some say the answer of what a covariate is depends on the context, so this gets a bit murky and confusing. See here for a pretty good explanation: theanalysisfactor.com/confusing-statistical-terms-5-covariate Commented May 24, 2019 at 14:37

From Wikipedia:

Depending on the context, an independent variable is sometimes called a "predictor variable", regressor, covariate, "controlled variable", "manipulated variable", "explanatory variable", exposure variable (see reliability theory), "risk factor" (see medical statistics), "feature" (in machine learning and pattern recognition) or "input variable." In econometrics, the term "control variable" is usually used instead of "covariate".

• Assume that you are solving linear regression, where you are trying to find a relation $$\textbf{y} = f(\textbf{X})$$. In this case, $$\textbf{X}$$ are independent variables and $$\textbf{y}$$ is the dependent variable.
• Typically, $$\textbf{X}$$ consists of multiple variables which may have some relations between them, i.e. they "co-vary" -- hence the term "covariate".

Let's take a concrete example. Suppose you wish to predict the price of a house in a neighborhood, $$\textbf{y}$$ using the following "co-variates", $$\textbf{X}$$:

• Width, $$x_1$$
• Breadth, $$x_2$$
• Number of floors, $$x_3$$
• Area of the house, $$x_4$$
• Distance to downtown, $$x_5$$
• Distance to hospital, $$x_6$$

For a linear regression problem, $$\textbf{y} = f(\textbf{X})$$ the price of the house is dependent on all co-variates, i.e. $$\textbf{y}$$ is dependent on $$\textbf{X}$$. Do any of the co-variates depend on the price of the house? In other words, is $$\textbf{X}$$ dependent on $$\textbf{y}$$? The answer is NO. Hence, $$\textbf{X}$$ is the independent variable and $$\textbf{y}$$ is the dependent variable. This encapsulates a cause and effect relationship. If the independent variable changes, its effect is seen on the dependent variable.

Now, are all the co-variates independent of each other? The answer is NO! A better answer is, well it depends!

The area of the house ($$x_4$$) is dependent on the width ($$x_1$$), breadth ($$x_2$$) and the number of floors ($$x_3$$), whereas, distances to downtown ($$x_5$$) and hospital ($$x_6$$) are independent of the area of the house ($$x_4$$).

Hope that helps!

• Thanks, I read that. So I guess what I said is true? I.e. covariates are random variables that have their outcomes observed, and such observation is used to predict the outcome of unseen random variables of interest? Commented May 23, 2019 at 22:20
• Thanks! (re your own text below the Wiki quote). So, can I say that if $X$ are independent variables, then each such variable is giving me information such that all of the information is unique (no other variable gives me), but in case they have some relations (i.e. not perfectly independent --- aka co-varying), then each variable is giving me information that is partly novel, but also partly also given to me by other variables? In other words, if I compress the random variables by a perfect compression, can I say I will have independent random variables (no co-vary)? Commented May 23, 2019 at 23:00
• When we call X as independent variables, they are independent with regards to thier relation with y. X can consist of variables that can be independent or dependent on each other. Ideally you want all variables in X to be independent, which can be achieved using something like PCA. Commented May 23, 2019 at 23:07
• THANKS THAT IS AMAZING. So I guess the name "independent" is from the perspective of cause-effect perspective, where we think that $X$ is what causes the price change, but me simply bumping the price of my house, it won't make my house suddenly become bigger? Is this the perspective from which "independent" is used? In other words, by "independent", they do not mean statistical independence as they are still statistically dependent. Am I right? Any mistakes? Commented May 24, 2019 at 14:05
• Note that independence doesn't imply a lack of a cause and effect relationship. See this post: stats.stackexchange.com/questions/357255/… Commented May 24, 2019 at 14:34

the way linear regression is generally run (there are ways to ask for different slope calculations) you are getting the unique impact of one predictor on the dependent variable. Its shared impact with other predictors on the DV (or indirect impact as with structural equation models I believe) is not part of the slope. it is sometimes stated that the slope is the impact of a specific predictor setting all other X to zero (although that breaks down obviously when some X can not take on the value of 0 or you have interaction).

In general terms, covariates are characteristics of the participants in an experiment. If you collect data on characteristics before you run an experiment, you could use that data to see how your treatment affects different groups or populations. Or, you could use that data to control for the influence of any covariate.

Covariates may affect the outcome in a study. For example, you are running an experiment to see how corn plants tolerate drought. Level of drought is the actual “treatment,” but it isn’t the only factor that affects how plants perform: size is a known factor that affects tolerance levels so that you would run plant size as a covariate.

Another example (from Penn State): Let’s say you are comparing the salaries of men and women to see who earns more. One factor that you need to control for is that people tend to earn more the longer they are out of college. Years out of college, in this case, is a covariate.

A covariate can be an independent variable (i.e., of direct interest) or it can be an unwanted, confounding variable. Adding a covariate to a model can increase the accuracy of your results.