Questions tagged [multiple-regression]

Regression that includes two or more non-constant independent variables.

Filter by
Sorted by
Tagged with
404 votes
7 answers

When conducting multiple regression, when should you center your predictor variables & when should you standardize them?

In some literature, I have read that a regression with multiple explanatory variables, if in different units, needed to be standardized. (Standardizing consists in subtracting the mean and dividing ...
mathieu_r's user avatar
  • 4,451
70 votes
2 answers

Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?

The coefficient of an explanatory variable in a multiple regression tells us the relationship of that explanatory variable with the dependent variable. All this, while 'controlling' for the other ...
Siddharth Gopi's user avatar
80 votes
5 answers

How can adding a 2nd IV make the 1st IV significant?

I have what is probably a simple question, but it is baffling me right now, so I am hoping you can help me out. I have a least squares regression model, with one independent variable and one ...
EvKohl's user avatar
  • 1,190
36 votes
3 answers

How to tell the difference between linear and non-linear regression models?

I was reading the following link on non linear regression SAS Non Linear. My understanding from reading the first section "Nonlinear Regression vs. Linear Regression" was that the equation below is ...
mHelpMe's user avatar
  • 657
81 votes
0 answers

How can a regression be significant yet all predictors be non-significant? [duplicate]

My multiple regression analysis model has a statistically significant F value however all beta values are statistically non-significant. All the regression assumptions are met. No multicollinearity ...
Serene's user avatar
  • 811
54 votes
3 answers

Why is polynomial regression considered a special case of multiple linear regression?

If polynomial regression models nonlinear relationships, how can it be considered a special case of multiple linear regression? Wikipedia notes that "Although polynomial regression fits a nonlinear ...
gavinmh's user avatar
  • 1,085
51 votes
2 answers

Multiple regression or partial correlation coefficient? And relations between the two

I don't even know if this question makes sense, but what is the difference between multiple regression and partial correlation (apart from the obvious differences between correlation and regression, ...
user34927's user avatar
  • 239
14 votes
2 answers

Bayesian logit model - intuitive explanation?

I must confess that I previously haven't heard of that term in any of my classes, undergrad or grad. What does it mean for a logistic regression to be Bayesian? I'm looking for an explanation with a ...
BCLC's user avatar
  • 2,394
54 votes
3 answers

Suppression effect in regression: definition and visual explanation/depiction

What is a suppressor variable in multiple regression and what might be the ways to display suppression effect visually (its mechanics or its evidence in results)? I'd like to invite everybody who has ...
ttnphns's user avatar
  • 56.6k
44 votes
4 answers

Significance contradiction in linear regression: significant t-test for a coefficient vs non-significant overall F-statistic

I'm fitting a multiple linear regression model between 4 categorical variables (with 4 levels each) and a numerical output. My dataset has 43 observations. Regression gives me the following $p$-...
Leo's user avatar
  • 2,584
27 votes
3 answers

How to model this odd-shaped distribution (almost a reverse-J)

My dependent variable shown below doesn't fit any stock distribution that I know of. Linear regression produces somewhat non-normal, right-skewed residuals that relate to predicted Y in an odd way (...
rolando2's user avatar
  • 12k
29 votes
2 answers

In what order should you do linear regression diagnostics?

In linear regression analysis, we analyze outliers, investigate multicollinearity, test heteroscedasticty. The question is: Is there any order to apply these? I mean, do we have to analyze outliers ...
halil's user avatar
  • 295
30 votes
1 answer

The proof of shrinking coefficients using ridge regression through "spectral decomposition"

I have understood how ridge regression shrinks coefficients towards zero geometrically. Moreover I know how to prove that in the special "Orthonormal Case," but I am confused how that works in the ...
jeza's user avatar
  • 1,989
12 votes
2 answers

Significant predictors become non-significant in multiple logistic regression

When I analyze my variables in two separate (univariate) logistic regression models, I get the following: ...
Annie's user avatar
  • 121
43 votes
5 answers

How to derive the least square estimator for multiple linear regression?

In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
Saber CN's user avatar
  • 809
67 votes
3 answers

What is the effect of having correlated predictors in a multiple regression model?

I learned in my linear models class that if two predictors are correlated and both are included in a model, one will be insignificant. For example, assume the size of a house and the number of ...
Vivek Subramanian's user avatar
17 votes
1 answer

How to choose between ANOVA and ANCOVA in a designed experiment?

I am conducting an experiment which has the following: DV: Slice consumption (continuous or could be categorical) IV: Healthy message, unhealthy message, no message (control) (3 groups in which ...
mobo's user avatar
  • 213
9 votes
3 answers

Multiple logistic regression power analysis

I have a logistic regression model and output an $R^2$ value. I then go and add another predictor variable to fit a second model. I can output a new $R^2$ value associated with the second model. When ...
lukeg's user avatar
  • 411
94 votes
7 answers

Explain the difference between multiple regression and multivariate regression, with minimal use of symbols/math

Are multiple and multivariate regression really different? What is a variate anyways?
Neil McGuigan's user avatar
31 votes
2 answers

Transforming variables for multiple regression in R

I am trying to perform a multiple regression in R. However, my dependent variable has the following plot: Here is a scatterplot matrix with all my variables (...
zgall1's user avatar
  • 869
30 votes
2 answers

Why are p-values misleading after performing a stepwise selection?

Let's consider for example a linear regression model. I heard that, in data mining, after performing a stepwise selection based on the AIC criterion, it is misleading to look at the p-values to test ...
John M's user avatar
  • 2,097
12 votes
1 answer

Data space, variable space, observation space, model space (e.g. in linear regression)

Suppose we have the data matrix $\mathbf{X}$, which is $n$-by-$p$, and the label vector $Y$, which is $n$-by-one. Here, each row of the matrix is an observation, and each column corresponds to a ...
user3813057's user avatar
  • 1,082
81 votes
2 answers

Multivariate multiple regression in R

I have 2 dependent variables (DVs) each of whose score may be influenced by the set of 7 independent variables (IVs). DVs are continuous, while the set of IVs consists of a mix of continuous and ...
Andrej's user avatar
  • 2,213
51 votes
5 answers

Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?

I just browsed through this wonderful book: Applied multivariate statistical analysis by Johnson and Wichern. The irony is, I am still not able to understand the motivation for using multivariate (...
KarthikS's user avatar
  • 1,156
32 votes
1 answer

Geometric interpretation of multiple correlation coefficient $R$ and coefficient of determination $R^2$

I am interested in the geometric meaning of the multiple correlation $R$ and coefficient of determination $R^2$ in the regression $y_i = \beta_1 + \beta_2 x_{2,i} + \dots + \beta_k x_{k,i} + \...
Silverfish's user avatar
  • 22.9k
30 votes
3 answers

What does "all else equal" mean in multiple regression?

When we do multiple regressions and say we are looking at the average change in the $y$ variable for a change in an $x$ variable, holding all other variables constant, what values are we holding the ...
EconStats's user avatar
  • 865
29 votes
2 answers

Interpretation of betas when there are multiple categorical variables

I understand the concept that $\hat\beta_0$ is the mean for when the categorical variable is equal to 0 (or is the reference group), giving the end interpretation that the regression coefficient is ...
Renee's user avatar
  • 425
15 votes
3 answers

Either quadratic or interaction term is significant in isolation, but neither are together

As part of an assignment, I had to fit a model with two predictor variables. I then had to draw a plot of the models' residuals against one of the included predictors and make changes based on that. ...
Tal Bashan's user avatar
63 votes
5 answers

Is adjusting p-values in a multiple regression for multiple comparisons a good idea?

Lets assume you are a social science researcher/econometrician trying to find relevant predictors of demand for a service. You have 2 outcome/dependent variables describing the demand (using the ...
Mikael M's user avatar
  • 733
58 votes
4 answers

How to visualize a fitted multiple regression model?

I am currently writing a paper with several multiple regression analyses. While visualizing univariate linear regression is easy via scatter plots, I was wondering whether there is any good way to ...
Shawn Wang's user avatar
  • 1,385
32 votes
4 answers

What are variable importance rankings useful for?

I have become somewhat of a nihilist when it comes to variable importance rankings (in the context of multivariate models of all kinds). Often in the course of my work, I am asked to either assist ...
Matthew Drury's user avatar
14 votes
1 answer

interpretation of betareg coef

I have a data that where the outcome is the proportion of a species observed in an area by a machine on 2 separate days. Since the outcome is a proportion and does not include 0 or 1 I used a beta ...
user3022875's user avatar
14 votes
2 answers

Positive correlation and negative regressor coefficient sign

Is it possible to obtain a positive correlation between a regressor and a response (+0,43) and, after that, obtain a negative coefficient in the fitted regression ...
Javier Bermejo's user avatar
10 votes
4 answers

Omitted variable bias: which predictors do I need to include, and why?

For a last couple of weeks I've been thinking about OVB (Omitted variable bias) in the context of regression and solution for that (how to avoid this problem). I am acquainted with Shalizi's lectures (...
Lil'Lobster's user avatar
  • 1,478
35 votes
2 answers

What does an Added Variable Plot (Partial Regression Plot) explain in a multiple regression?

I have a model of Movies dataset and I used the regression: ...
Abhishek Choudhary's user avatar
31 votes
3 answers

Does the order of explanatory variables matter when calculating their regression coefficients?

At first I thought the order didn’t matter, but then I read about the gram-schmidt orthogonalization process for calculating multiple regression coefficients, and now I’m having second thoughts. ...
Ryan Zotti's user avatar
  • 6,297
25 votes
2 answers

What is the correct way to test for significant differences between coefficients?

I'm hoping someone can help straighten out a point of confusion for me. Say I want to test whether 2 sets of regression coefficients are significantly different from each other, with the following set ...
cashoes's user avatar
  • 523
21 votes
3 answers

Does adding more variables into a multivariable regression change coefficients of existing variables?

Say I have a multivariable (several independent variables) regression that consists of 3 variables. Each of those variables has a given coefficient. If I decide to introduce a 4th variable and rerun ...
Lukas Pleva's user avatar
4 votes
2 answers

Can I analyze or model a conditional correlation?

In my research I'm looking at the correlation between self-harm and aggression (both continuous). Now, I also have some variables (e.g. depressive symptoms; also continuous) which I do believe ...
Joël Derks's user avatar
2 votes
1 answer

Correction for multiple testing in Multiple regression analysis

Correction for multiple testing (e.g. Bonferroni correction) is recommended when multiple statistical tests are performed on same data. In multiple regression analysis, multiple testing is integral ...
rnso's user avatar
  • 9,547
34 votes
4 answers

Importance of predictors in multiple regression: Partial $R^2$ vs. standardized coefficients

I am wondering what the exact relationship between partial $R^2$ and coefficients in a linear model is and whether I should use only one or both to illustrate the importance and influence of factors. ...
robert's user avatar
  • 746
16 votes
3 answers

Estimating $b_1 x_1+b_2 x_2$ instead of $b_1 x_1+b_2 x_2+b_3x_3$

I have a theoretical economic model which is as follows, $$ y = a + b_1x_1 + b_2x_2 + b_3x_3 + u $$ So theory says that there are $x_1$, $x_2$ and $x_3$ factors to estimate $y$. Now I have the real ...
renathy's user avatar
  • 457
32 votes
3 answers

How to deal with multicollinearity when performing variable selection?

I have a dataset with 9 continuous independent variables. I'm trying to select amongst these variables to fit a model to a single percentage (dependent) variable, ...
Julie's user avatar
  • 811
15 votes
1 answer

Should I run separate regressions for every community, or can community simply be a controlling variable in an aggregated model?

I am running an OLS model with a continuous asset index variable as the DV. My data is aggregated from three similar communities in close geographic proximity to one another. Despite this, I thought ...
cadamt's user avatar
  • 309
15 votes
2 answers

What is an unbiased estimate of population R-square?

I am interested in getting an unbiased estimate of $R^2$ in a multiple linear regression. On reflection, I can think of two different values that an unbiased estimate of $R^2$ might be trying to ...
Jeromy Anglim's user avatar
8 votes
4 answers

Why ANOVA/Regression results change when controlling for another variable

This question might be very basic, but somehow I don't understand this point. Suppose initially I used a univariate regression equation such as GDP=a+b*Income ...
Beta's user avatar
  • 6,284
7 votes
3 answers

Why is the intercept in multiple regression changing when including/excluding regressors?

I have a seemingly naive question regarding the interpretation of the intercept in multiple regression. What I found several times is something like this: The constant/intercept is defined as the ...
Marco's user avatar
  • 316
51 votes
6 answers

Choosing variables to include in a multiple linear regression model

I am currently working to build a model using a multiple linear regression. After fiddling around with my model, I am unsure how to best determine which variables to keep and which to remove. My ...
cryptic_star's user avatar
  • 1,127
26 votes
1 answer

How to deal with high correlation among predictors in multiple regression?

I found a reference in an article that goes like: According to Tabachnick & Fidell (1996) the independent variables with a bivariate correlation more than .70 should not be included in ...
Ander's user avatar
  • 301
24 votes
3 answers

How to split r-squared between predictor variables in multiple regression?

I have just read a paper in which the authors carried out a multiple regression with two predictors. The overall r-squared value was 0.65. They provided a table which split the r-squared between the ...
luciano's user avatar
  • 14k

2 3 4 5