8
$\begingroup$

I do realise (some of) this has already been addressed here (e.g., Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?, Explain the difference between multiple regression and multivariate regression, with minimal use of symbols/math, and Defining a univariate regression). Nevertheless, I hope I can get some more insight on this.

What are the differences among the terms in the title? Does this vary (and therefore causes confusion)? Or is there one proper terminology, and some people just use it incorrectly?

  1. For example, from what I understand, simple (linear) regression would be where we have one response and one explanatory variable. Multiple (linear) regression is when we have one response and multiple explanatory variables. So far so good - I'm not confused here yet.

  2. But when one says "multivariate", does this imply multiple RESPONSES, or multiple EXPLANATORY VARIABLES (or both)? I assume the former (also judging from the links provided), but people seem to use these interchangeably. By extension, "univariate" would likely mean a single response?

  3. Further, the term "multivariate analyses" (PCA, discriminant analysis, etc.) usually implies multiple responses, right?

  4. And finally, the term "bivariate". Zuur et al. 2009 (Mixed effects models and extensions in ecology with R) state that:

    The bivariate linear regression model is defined by: $$Y_i = α + β × X_i + ε_i \\ \text{ where }ε_i \sim N(0, σ^2)$$

    This seems to refer to one response and one predictor (i.e. simple linear regression). So, in this case, does the term "bivariate" refer to two variables in total (one response, one predictor)?

Is there some standard terminology or is it just a mess?

Improve this question
$\endgroup$
1
  • $\begingroup$ I think that multivariate linear regression is used when we have more than one response, i.e. $$Y_i = X_i\beta + \epsilon$$ where $Y \in R^{1xq}$ is a vector of $q$ responses and $\beta \in R^{pxq}$ is a matrix of estimated coefficients $\endgroup$
    – Łukasz Grad
    Commented Feb 15, 2017 at 14:57

2 Answers 2

Reset to default
8
$\begingroup$

As for Question 1, you are correct with what you said.

As for Question 2, multivariate stands for an analysis involving more than one response variables. To my knowledge there is no differentiation in terminology with respect to the predictor variables. To be consistent one could maybe say, but I am not sure, "simple multivariate regression" when multiple responses and one predictor variable are present.

As for Question 3, I'd say you are right again.

As for Question 4, the term bivariate refers to a situation when there are two continuous variables in total, i.e. an analysis that can be visualized in a 2d scatter plot (simple linear regression and correlation for example).

So now what does univariate refers too? I think (and I might be wrong) that is the case when you have one response and one or more categorical predictor(s). So, for example you measure the heights of trees coming from the same parent tree, or the weight of chicken fed with the different feeds. This type of analyses would be analyzed as a t-test or Analysis of Variance.

The difference between univariate and bivariate can be seen when you visualize the data. If you plot something as a bar graph, (or dot plot) it is univariate, if you plot something on a 2d scatter plot, it is bivariate. I might be wrong here but I am sure if that's the case someone will comment!

Improve this answer
$\endgroup$
3
  • $\begingroup$ Any other thoughts anyone (either agreeing, disagreeing, adding something)? $\endgroup$
    – Tilen
    Commented Feb 18, 2017 at 9:21
  • $\begingroup$ I was wondering if one could also use the term "univariable" instead of "simple" for a regression model with a single predictor variable? [I was just dealing with a quantile regression with a single predictor variable without intercept and a single dependent variable and was wondering what term I should use for that - simple quantile regression or univariable quantile regression] $\endgroup$
    – Tom Wenseleers
    Commented Mar 8, 2019 at 9:52
  • $\begingroup$ @TomWenseleers Yes I see what you mean and think there is no reason why not. In fact some digging on this site revealed: stats.stackexchange.com/questions/172868/… where this is discussed. $\endgroup$
    – Stefan
    Commented Mar 8, 2019 at 14:55
4
$\begingroup$

Let $y$ be a predicted variable and let $x$ be a predictor variable.

  1. One $y$ and one $x$ = simple regression

  2. One $y$ and many $x$ = multiple regression

  3. Many $y$ and one $x$ = multivariate simple regression

  4. Many $y$ and many $x$ = multivariate multiple regression

In practice, because these four cases can all be, and usually are, handled within the same framework (e.g., general linear modeling), the differences between them are purely terminological and not statistical. However, using these terms consistently can enhance communication.

David A. Freedman (2009). Statistical Models: Theory and Practice. Cambridge University Press.

Rencher, Alvin C.; Christensen, William F. (2012), "Chapter 10, Multivariate regression", Methods of Multivariate Analysis, Wiley Series in Probability and Statistics (3rd ed.), John Wiley & Sons, p. 19.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Would this terminology also extend to other types of regression with other losses than least squares? E.g. quantile regression? $\endgroup$
    – Tom Wenseleers
    Commented Mar 8, 2019 at 9:54
  • $\begingroup$ I think it could. With one y and many x, you could call that multiple quantile regression. But by providing a description as well as a name would be clearest of all. $\endgroup$
    – Jeffrey Girard
    Commented Mar 8, 2019 at 12:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.