# Questions tagged [multiple-regression]

Regression that includes two or more non-constant independent variables.

699 questions
Filter by
Sorted by
Tagged with
404 votes
7 answers
407k views

### When conducting multiple regression, when should you center your predictor variables & when should you standardize them?

In some literature, I have read that a regression with multiple explanatory variables, if in different units, needed to be standardized. (Standardizing consists in subtracting the mean and dividing ...
• 4,451
70 votes
2 answers
26k views

### Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?

The coefficient of an explanatory variable in a multiple regression tells us the relationship of that explanatory variable with the dependent variable. All this, while 'controlling' for the other ...
• 1,545
80 votes
5 answers
40k views

### How can adding a 2nd IV make the 1st IV significant?

I have what is probably a simple question, but it is baffling me right now, so I am hoping you can help me out. I have a least squares regression model, with one independent variable and one ...
• 1,190
36 votes
3 answers
27k views

### How to tell the difference between linear and non-linear regression models?

I was reading the following link on non linear regression SAS Non Linear. My understanding from reading the first section "Nonlinear Regression vs. Linear Regression" was that the equation below is ...
• 657
81 votes
0 answers
64k views

### How can a regression be significant yet all predictors be non-significant? [duplicate]

My multiple regression analysis model has a statistically significant F value however all beta values are statistically non-significant. All the regression assumptions are met. No multicollinearity ...
• 811
54 votes
3 answers
36k views

### Why is polynomial regression considered a special case of multiple linear regression?

If polynomial regression models nonlinear relationships, how can it be considered a special case of multiple linear regression? Wikipedia notes that "Although polynomial regression fits a nonlinear ...
• 1,085
51 votes
2 answers
70k views

### Multiple regression or partial correlation coefficient? And relations between the two

I don't even know if this question makes sense, but what is the difference between multiple regression and partial correlation (apart from the obvious differences between correlation and regression, ...
• 239
14 votes
2 answers
24k views

### Bayesian logit model - intuitive explanation?

I must confess that I previously haven't heard of that term in any of my classes, undergrad or grad. What does it mean for a logistic regression to be Bayesian? I'm looking for an explanation with a ...
• 2,394
54 votes
3 answers
91k views

### Suppression effect in regression: definition and visual explanation/depiction

What is a suppressor variable in multiple regression and what might be the ways to display suppression effect visually (its mechanics or its evidence in results)? I'd like to invite everybody who has ...
• 56.6k
44 votes
4 answers
44k views

### Significance contradiction in linear regression: significant t-test for a coefficient vs non-significant overall F-statistic

I'm fitting a multiple linear regression model between 4 categorical variables (with 4 levels each) and a numerical output. My dataset has 43 observations. Regression gives me the following $p$-...
• 2,584
27 votes
3 answers
7k views

### How to model this odd-shaped distribution (almost a reverse-J)

My dependent variable shown below doesn't fit any stock distribution that I know of. Linear regression produces somewhat non-normal, right-skewed residuals that relate to predicted Y in an odd way (...
• 12k
29 votes
2 answers
5k views

### In what order should you do linear regression diagnostics?

In linear regression analysis, we analyze outliers, investigate multicollinearity, test heteroscedasticty. The question is: Is there any order to apply these? I mean, do we have to analyze outliers ...
• 295
30 votes
1 answer
17k views

### The proof of shrinking coefficients using ridge regression through "spectral decomposition"

I have understood how ridge regression shrinks coefficients towards zero geometrically. Moreover I know how to prove that in the special "Orthonormal Case," but I am confused how that works in the ...
• 1,989
12 votes
2 answers
42k views

### Significant predictors become non-significant in multiple logistic regression

When I analyze my variables in two separate (univariate) logistic regression models, I get the following: ...
• 121
43 votes
5 answers
138k views

### How to derive the least square estimator for multiple linear regression?

In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
• 809
67 votes
3 answers
131k views

### What is the effect of having correlated predictors in a multiple regression model?

I learned in my linear models class that if two predictors are correlated and both are included in a model, one will be insignificant. For example, assume the size of a house and the number of ...
• 3,193
17 votes
1 answer
26k views

### How to choose between ANOVA and ANCOVA in a designed experiment?

I am conducting an experiment which has the following: DV: Slice consumption (continuous or could be categorical) IV: Healthy message, unhealthy message, no message (control) (3 groups in which ...
• 213
9 votes
3 answers
20k views

### Multiple logistic regression power analysis

I have a logistic regression model and output an $R^2$ value. I then go and add another predictor variable to fit a second model. I can output a new $R^2$ value associated with the second model. When ...
• 411
94 votes
7 answers
185k views

### Explain the difference between multiple regression and multivariate regression, with minimal use of symbols/math

Are multiple and multivariate regression really different? What is a variate anyways?
• 9,772
31 votes
2 answers
95k views

### Transforming variables for multiple regression in R

I am trying to perform a multiple regression in R. However, my dependent variable has the following plot: Here is a scatterplot matrix with all my variables (...
• 869
30 votes
2 answers
9k views

### Why are p-values misleading after performing a stepwise selection?

Let's consider for example a linear regression model. I heard that, in data mining, after performing a stepwise selection based on the AIC criterion, it is misleading to look at the p-values to test ...
• 2,097
12 votes
1 answer
9k views

### Data space, variable space, observation space, model space (e.g. in linear regression)

Suppose we have the data matrix $\mathbf{X}$, which is $n$-by-$p$, and the label vector $Y$, which is $n$-by-one. Here, each row of the matrix is an observation, and each column corresponds to a ...
• 1,082
81 votes
2 answers
78k views

### Multivariate multiple regression in R

I have 2 dependent variables (DVs) each of whose score may be influenced by the set of 7 independent variables (IVs). DVs are continuous, while the set of IVs consists of a mix of continuous and ...
• 2,213
51 votes
5 answers
30k views

### Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?

I just browsed through this wonderful book: Applied multivariate statistical analysis by Johnson and Wichern. The irony is, I am still not able to understand the motivation for using multivariate (...
• 1,156
32 votes
1 answer
14k views

• 44.3k
8 votes
4 answers
4k views

### Why ANOVA/Regression results change when controlling for another variable

This question might be very basic, but somehow I don't understand this point. Suppose initially I used a univariate regression equation such as GDP=a+b*Income ...
• 6,284
7 votes
3 answers
7k views

### Why is the intercept in multiple regression changing when including/excluding regressors?

I have a seemingly naive question regarding the interpretation of the intercept in multiple regression. What I found several times is something like this: The constant/intercept is defined as the ...
• 316
51 votes
6 answers
156k views

### Choosing variables to include in a multiple linear regression model

I am currently working to build a model using a multiple linear regression. After fiddling around with my model, I am unsure how to best determine which variables to keep and which to remove. My ...
• 1,127
26 votes
1 answer
60k views

### How to deal with high correlation among predictors in multiple regression?

I found a reference in an article that goes like: According to Tabachnick & Fidell (1996) the independent variables with a bivariate correlation more than .70 should not be included in ...
• 301
24 votes
3 answers
42k views

### How to split r-squared between predictor variables in multiple regression?

I have just read a paper in which the authors carried out a multiple regression with two predictors. The overall r-squared value was 0.65. They provided a table which split the r-squared between the ...
• 14k