Skip to main content
47 votes

Explain the difference between multiple regression and multivariate regression, with minimal use of symbols/math

Simple regression pertains to one dependent variable ($y$) and one independent variable ($x$): $y = f(x)$ Multiple regression (aka multivariable regression) pertains to one dependent variable and ...
stackoverflowuser2010's user avatar
30 votes
Accepted

Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?

Be sure to read the full example on the UCLA site that you linked. Regarding 1: Using a multivariate model helps you (formally, inferentially) compare coefficients across outcomes. In that linked ...
civilstat's user avatar
  • 4,431
14 votes
Accepted

Multivariate linear regression with lasso in r

For multivariate responses (number of dependent variables larger than 1), you need family = "mgaussian" in the call of glmnet. ...
NRH's user avatar
  • 18.5k
12 votes

Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?

Think about all the false and sometimes dangerous conclusions that come from simply multiplying probabilities, thinking events are independent. Because of all the built-in redundant safeguards, we ...
Michael R. Chernick's user avatar
12 votes

gaussian process regression for large datasets

You asked: in the case where 𝑛 is 10's of millions does Gaussian process regression still work? Not in the standard sense of constructing and inverting a large matrix. You have two options: 1)...
James Hensman's user avatar
11 votes

Explain the difference between multiple regression and multivariate regression, with minimal use of symbols/math

I think the key insight (and differentiator) here aside from the number of variables on either side of the equation is that for the case of multivariate regression, the goal is to utilize the fact ...
thecity2's user avatar
  • 1,955
9 votes

Neural network for multiple output regression

At first I thought generic_user's comment was a show-stopper, but I just realized it isn't: If I train d different networks on d different outputs, then each one will be fit to that dimension with no ...
Pavel Komarov's user avatar
9 votes

gaussian process regression for large datasets

There are a wide range of approaches to scale GPs to large datasets, for example: Low Rank Approaches: these endeavoring to create a low rank approximation to the covariance matrix. Most famously ...
j__'s user avatar
  • 2,362
8 votes

Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?

Consider this quote from p. 36 of Darcy Olsen's book The Right to Try [1]: But about sixteen weeks after the [eteplirsen] infusions began, Jenn started noticing changes in [her son] Max. "The kid ...
David C. Norris's user avatar
8 votes

Neural network for multiple output regression

A neural net with multiple outcomes takes the form $$ \mathbf{Y} = \mathbf{\gamma} + \mathbf{V}_1\Gamma_1 + \epsilon\\ \mathbf{V}_1 = a\left(\gamma_2 +\mathbf{V}_2\Gamma_2\right)\\ \mathbf{V}_2 = a\...
generic_user's user avatar
  • 13.5k
8 votes
Accepted

Simple, multiple, univariate, bivariate, multivariate - terminology

As for Question 1, you are correct with what you said. As for Question 2, multivariate stands for an analysis involving more than one response variables. To my knowledge there is no differentiation ...
Stefan's user avatar
  • 6,501
7 votes

Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?

Let's make a simple analogy, since that's all I can really try to contribute. Instead of univariate versus multivariate regression, let's consider univariate (marginal) versus multivariate (joint) ...
Wayne's user avatar
  • 21.5k
7 votes
Accepted

Multivariate linear regression vs. several univariate regression models

In the setting of classical multivariate linear regression, we have the model: $$Y = X \beta + \epsilon$$ where $X$ represents the independent variables, $Y$ represents multiple response variables, ...
user20160's user avatar
  • 32.9k
6 votes

What's the point in neural networks for multivariate regression?

Here's a cartoon representation of the models. Model A uses a single network to predict both outputs. Model B uses separate networks to predict both outputs. Because the input to both networks is the ...
user20160's user avatar
  • 32.9k
6 votes
Accepted

Minimum number of observation in multivariate regression

I believe that it is better to think in terms of power to detect effects of interest, rather than rules of thumb. That said, if you want a quick-and-dirty baseline value, the standard rule of thumb ...
gung - Reinstate Monica's user avatar
6 votes

Univariate approach to a Bivariate logistic regression

This should be possible with bivariate probit regression. The observed bivariate binary response can be represented via thresholding a bivariate normal latent variable. A bivariate normal with ...
kjetil b halvorsen's user avatar
5 votes
Accepted

Defining a univariate regression

Yes. "Multivariate" and "univariate", when they're used to describe models, refer to the number of dependent variables, not the number of independent variables. A multivariate linear regression model, ...
Kodiologist's user avatar
  • 20.4k
5 votes
Accepted

AIC Model Fitting - Multiple Models

A stepwise procedure using AIC is a very bad idea as it is equivalent to variable selection/elimination based on a p-value. A much better method is using expert knowledge to choose variables that ...
Robert Long's user avatar
  • 63.8k
5 votes

A reasonable multivariate regression error metric

Consider that you have m outputs. These outputs may be interpreted as a point in the m dimensional space. Then, for each ...
Hossein's user avatar
  • 3,494
5 votes

gaussian process regression for large datasets

Usually, what you can do is to train Gaussian Processes on subsamples of your dataset (bagging). Bagging is implemented in sk learn and can be used easily. See per example the documentation. Calling $...
RUser4512's user avatar
  • 10.4k
5 votes

How do I make sense of 200 regression models?

If I understood your data correctly, one method would be calculating feature importances for each model and maybe plotting them. Below is an example beeswarm plot and code in ...
kkz's user avatar
  • 213
5 votes

Model selection for multivariate mixed models

I usually take the view that the random structure (and the fixed structure) should be dictated by expert knowledge, not a statistical procedure. Of course there is nothing wrong with using an ...
Robert Long's user avatar
  • 63.8k
4 votes
Accepted

Linear regression with several DVs with correlated errors

This is in fact called multivariate regression. I just think that it's not a commonly-used term because it's not a commonly-used model. Note also that it is very easy to confuse with "multiple linear ...
shadowtalker's user avatar
  • 12.8k
4 votes

Explain the difference between multiple regression and multivariate regression, with minimal use of symbols/math

In multivariate regression there are more than one dependent variable with different variances (or distributions). The predictor variables may be more than one or multiple. So it is may be a multiple ...
Bhabesh Mahanta's user avatar
4 votes

Simple, multiple, univariate, bivariate, multivariate - terminology

Let $y$ be a predicted variable and let $x$ be a predictor variable. One $y$ and one $x$ = simple regression One $y$ and many $x$ = multiple regression Many $y$ and one $x$ = multivariate simple ...
Jeffrey Girard's user avatar
4 votes
Accepted

Is there a way to specify reduced-rank regression using $\mathbf{y} = \mathbf{X}\boldsymbol\beta + \boldsymbol\epsilon$?

In usual multiple regression the response variable $y$ is 1-dimensional so for each sample we can write an equation $$y = \boldsymbol \beta^\top \mathbf x + \epsilon,\tag{1a}$$ where $\mathbf x$ is an ...
amoeba's user avatar
  • 106k
4 votes

Logistic regression with multiple outcome variables (all categorical)

If the 5 outcome variables are meant to measure the same latent concept, the questions you have to ask is: From a research perspective, are you interested in assessing the impact of the predictor ...
Isabella Ghement's user avatar
4 votes
Accepted

Binary logistic regression with two dependent variables

You are using it wrong, there are three possible input formats for logistic regression in R As a factor: ‘success’ is interpreted as the factor not having the first level (and hence usually of ...
Tim's user avatar
  • 140k
4 votes
Accepted

Anyone know Multivariate OLS on Statsmodels?

At the time of writing this (Aug-2019) there is no MultivariateOLS in actual terms. That's why the _ infront of the call; it ...
usεr11852's user avatar
  • 44.9k
4 votes
Accepted

Linear regression with vector outputs

This is called general linear model (not to be confused with the generalized one) also known as multivariate linear regression model. $$ \mathbf{ Y = X B + \varepsilon } $$ where all $\mathbf{Y}$, $\...
Tim's user avatar
  • 140k

Only top scored, non community-wiki answers of a minimum length are eligible