Difference between Univariate Linear Regression and Simple Linear Regression?

Question

Is there any difference between Univariate Linear Regression and Simple Linear Regression? If so, what is the difference exactly? It seems both of them are exactly same. I would appreciate if anyone could cite a scientific paper that defines Univariate Linear Regression.

Answer 1

A good start (I hope uncontentious) on this is simply to note that univariate, bivariate and multivariate denote focus on one, two or many variables respectively. (Other words such as trivariate can be found but seem much more rarely used and rarely needed.)

Then it might follow that univariate techniques are those for which only one variable is needed (e.g. calculating mean or median, drawing a histogram); bivariate techniques are those which two variables are needed (e.g. correlation); while multivariate techniques are those for which many variables are needed, or (similarly if not identically) those which can be applied to many variables at once. Many and two sometimes overlap as when principal component analysis could be applied to two variables, not just many. (Or even one...)

I'd say that regression without qualification implies just one outcome or response variable, whereas multivariate regression implies two or more outcome or response variables (although regression with one predictor could naturally be seen as a special case).

Historically multiple regression has been used to refer to any set-up with several predictor variables and in contrast simple regression can be used when only one predictor is used. I'd assert, as a mixture of personal impressions from reading and personal opinions about good terminology, that references to multiple regression are fading slowly (it's routine in many fields, scientifically, statistically and computationally, and long since not very special) and that neither term, multiple or simple, fills an important gap.

A clear distinction in literature between multiple and multivariate regression doesn't stop multivariate being misused for multiple, a common confusion often seen on this site.

So much context, now the question:

The expression univariate regression seems to have crept into informal discussions fairly recently. Can anyone cite a good textbook reference? (*) I'd presume that it could only mean one outcome, one predictor. So it's at best redundant or a term for the simplest kind of regression. It's also at odds with the idea that such regression is a bivariate technique.

(*) The original question over 5 years ago as I write asked for scientific paper references, which no-one seems to have provided. I don't have one either.

Answer 2

Purist view: In general, there is clear difference between those two terms.

Univariate might have e.g. two explanatory variables (or five etc.) but always just one response variable. But it can also have just one expl. variable. In that scenario it is a simple linear regression as well.

Now, the question remains whether simple linear regression can have multiple response variables. According to Wikipedia (citing multiple text books), no, although not stated explicitly. (Subquestion: Can regression have more response than explanatory variables?)

Practical view: If you have one response variable (univariate) and multiple explanatory ones, you should call it (univariate) multiple regression. Wherever you have more than one variable, you need to be explicit about it. Otherwise one variable is expected, thus the term univariate is redundant.

In conclusion, univariate r. and simple linear regression describe (most likely) the same kind of regression, but in most cases it will not make sense to use the former term. (The opposite is only true when you contrast it with multivariate regression.)

Answer 3

Simple Linear Regression is defined in as model with a single explanatory variable (i.e., the independent variable).

According to this answer,, Univariate Linear Regression refers to a model with a single response variable (i.e., the dependent variable). This answer corroborates the theory.

Now, here is a claim that says Simple regression necessarily has a single dependent variable too, but I cannot verify the claim. A model with one explanatory variable and more than one response variable will still be called simple (and multivariate), I think.

I have seen the terms 'Simple' and 'Univariate' used interchangeably, and was under the impression that there is no difference. But I reckon it is best to keep that distinction.

Answer 4

Univariate Model :

1. Simple : 1 DV & 1 IV

2. Multiple: 1 DV & many IV

Multivariate Model:

1. Simple Multivariate: many DV & 1 IV
2. Multiple Multivariate: many DV & many IV