Questions tagged [multiple-regression]
Regression that includes two or more non-constant independent variables.
699
questions
404
votes
7
answers
407k
views
When conducting multiple regression, when should you center your predictor variables & when should you standardize them?
In some literature, I have read that a regression with multiple explanatory variables, if in different units, needed to be standardized. (Standardizing consists in subtracting the mean and dividing ...
- 4,451
70
votes
2
answers
26k
views
Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?
The coefficient of an explanatory variable in a multiple regression tells us the relationship of that explanatory variable with the dependent variable. All this, while 'controlling' for the other ...
- 1,545
80
votes
5
answers
40k
views
How can adding a 2nd IV make the 1st IV significant?
I have what is probably a simple question, but it is baffling me right now, so I am hoping you can help me out.
I have a least squares regression model, with one independent variable and one ...
- 1,190
36
votes
3
answers
27k
views
How to tell the difference between linear and non-linear regression models?
I was reading the following link on non linear regression SAS Non Linear. My understanding from reading the first section "Nonlinear Regression vs. Linear Regression" was that the equation below is ...
- 657
81
votes
0
answers
64k
views
How can a regression be significant yet all predictors be non-significant? [duplicate]
My multiple regression analysis model has a statistically significant F value however all beta values are statistically non-significant.
All the regression assumptions are met. No multicollinearity ...
- 811
54
votes
3
answers
36k
views
Why is polynomial regression considered a special case of multiple linear regression?
If polynomial regression models nonlinear relationships, how can it be considered a special case of multiple linear regression?
Wikipedia notes that "Although polynomial regression fits a nonlinear ...
- 1,085
51
votes
2
answers
70k
views
Multiple regression or partial correlation coefficient? And relations between the two
I don't even know if this question makes sense, but what is the difference between multiple regression and partial correlation (apart from the obvious differences between correlation and regression, ...
- 239
14
votes
2
answers
24k
views
Bayesian logit model - intuitive explanation?
I must confess that I previously haven't heard of that term in any of my classes, undergrad or grad.
What does it mean for a logistic regression to be Bayesian? I'm looking for an explanation with a ...
- 2,394
54
votes
3
answers
91k
views
Suppression effect in regression: definition and visual explanation/depiction
What is a suppressor variable in multiple regression and what might be the ways to display suppression effect visually (its mechanics or its evidence in results)? I'd like to invite everybody who has ...
- 56.6k
44
votes
4
answers
44k
views
Significance contradiction in linear regression: significant t-test for a coefficient vs non-significant overall F-statistic
I'm fitting a multiple linear regression model between 4 categorical variables (with 4 levels each) and a numerical output. My dataset has 43 observations.
Regression gives me the following $p$-...
- 2,584
27
votes
3
answers
7k
views
How to model this odd-shaped distribution (almost a reverse-J)
My dependent variable shown below doesn't fit any stock distribution that I know of. Linear regression produces somewhat non-normal, right-skewed residuals that relate to predicted Y in an odd way (...
- 12k
29
votes
2
answers
5k
views
In what order should you do linear regression diagnostics?
In linear regression analysis, we analyze outliers, investigate multicollinearity, test heteroscedasticty.
The question is: Is there any order to apply these? I mean, do we have to analyze outliers ...
- 295
30
votes
1
answer
17k
views
The proof of shrinking coefficients using ridge regression through "spectral decomposition"
I have understood how ridge regression shrinks coefficients towards zero geometrically.
Moreover I know how to prove that in the special "Orthonormal Case," but I am confused how that works in the ...
- 1,989
12
votes
2
answers
42k
views
Significant predictors become non-significant in multiple logistic regression
When I analyze my variables in two separate (univariate) logistic regression models, I get the following:
...
- 121
43
votes
5
answers
138k
views
How to derive the least square estimator for multiple linear regression?
In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
- 809
67
votes
3
answers
131k
views
What is the effect of having correlated predictors in a multiple regression model?
I learned in my linear models class that if two predictors are correlated and both are included in a model, one will be insignificant. For example, assume the size of a house and the number of ...
- 3,193
17
votes
1
answer
26k
views
How to choose between ANOVA and ANCOVA in a designed experiment?
I am conducting an experiment which has the following:
DV: Slice consumption (continuous or could be categorical)
IV: Healthy message, unhealthy message, no message (control) (3 groups in which ...
- 213
9
votes
3
answers
20k
views
Multiple logistic regression power analysis
I have a logistic regression model and output an $R^2$ value. I then go and add another predictor variable to fit a second model. I can output a new $R^2$ value associated with the second model. When ...
- 411
94
votes
7
answers
185k
views
Explain the difference between multiple regression and multivariate regression, with minimal use of symbols/math
Are multiple and multivariate regression really different? What is a variate anyways?
- 9,772
31
votes
2
answers
95k
views
Transforming variables for multiple regression in R
I am trying to perform a multiple regression in R. However, my dependent variable has the following plot:
Here is a scatterplot matrix with all my variables (...
- 869
30
votes
2
answers
9k
views
Why are p-values misleading after performing a stepwise selection?
Let's consider for example a linear regression model. I heard that, in data mining, after performing a stepwise selection based on the AIC criterion, it is misleading to look at the p-values to test ...
- 2,097
12
votes
1
answer
9k
views
Data space, variable space, observation space, model space (e.g. in linear regression)
Suppose we have the data matrix $\mathbf{X}$, which is $n$-by-$p$, and the label vector $Y$, which is $n$-by-one. Here, each row of the matrix is an observation, and each column corresponds to a ...
- 1,082
81
votes
2
answers
78k
views
Multivariate multiple regression in R
I have 2 dependent variables (DVs) each of whose score may be influenced by the set of 7 independent variables (IVs). DVs are continuous, while the set of IVs consists of a mix of continuous and ...
- 2,213
51
votes
5
answers
30k
views
Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?
I just browsed through this wonderful book: Applied multivariate statistical analysis by Johnson and Wichern. The irony is, I am still not able to understand the motivation for using multivariate (...
- 1,156
32
votes
1
answer
14k
views
Geometric interpretation of multiple correlation coefficient $R$ and coefficient of determination $R^2$
I am interested in the geometric meaning of the multiple correlation $R$ and coefficient of determination $R^2$ in the regression $y_i = \beta_1 + \beta_2 x_{2,i} + \dots + \beta_k x_{k,i} + \...
- 22.9k
30
votes
3
answers
18k
views
What does "all else equal" mean in multiple regression?
When we do multiple regressions and say we are looking at the average change in the $y$ variable for a change in an $x$ variable, holding all other variables constant, what values are we holding the ...
- 865
29
votes
2
answers
40k
views
Interpretation of betas when there are multiple categorical variables
I understand the concept that $\hat\beta_0$ is the mean for when the categorical variable is equal to 0 (or is the reference group), giving the end interpretation that the regression coefficient is ...
- 425
15
votes
3
answers
13k
views
Either quadratic or interaction term is significant in isolation, but neither are together
As part of an assignment, I had to fit a model with two predictor variables. I then had to draw a plot of the models' residuals against one of the included predictors and make changes based on that. ...
- 151
63
votes
5
answers
49k
views
Is adjusting p-values in a multiple regression for multiple comparisons a good idea?
Lets assume you are a social science researcher/econometrician trying to find relevant predictors of demand for a service. You have 2 outcome/dependent variables describing the demand (using the ...
- 733
58
votes
4
answers
118k
views
How to visualize a fitted multiple regression model?
I am currently writing a paper with several multiple regression analyses. While visualizing univariate linear regression is easy via scatter plots, I was wondering whether there is any good way to ...
- 1,385
32
votes
4
answers
7k
views
What are variable importance rankings useful for?
I have become somewhat of a nihilist when it comes to variable importance rankings (in the context of multivariate models of all kinds).
Often in the course of my work, I am asked to either assist ...
- 35.2k
14
votes
1
answer
7k
views
interpretation of betareg coef
I have a data that where the outcome is the proportion of a species observed in an area by a machine on 2 separate days. Since the outcome is a proportion and does not include 0 or 1 I used a beta ...
- 796
14
votes
2
answers
28k
views
Positive correlation and negative regressor coefficient sign
Is it possible to obtain a positive correlation between a regressor and a response (+0,43) and, after that, obtain a negative coefficient in the fitted regression ...
- 231
10
votes
4
answers
8k
views
Omitted variable bias: which predictors do I need to include, and why?
For a last couple of weeks I've been thinking about OVB (Omitted variable bias) in the context of regression and solution for that (how to avoid this problem). I am acquainted with Shalizi's lectures (...
- 1,478
35
votes
2
answers
49k
views
What does an Added Variable Plot (Partial Regression Plot) explain in a multiple regression?
I have a model of Movies dataset and I used the regression:
...
31
votes
3
answers
12k
views
Does the order of explanatory variables matter when calculating their regression coefficients?
At first I thought the order didn’t matter, but then I read about the gram-schmidt orthogonalization process for calculating multiple regression coefficients, and now I’m having second thoughts.
...
- 6,297
25
votes
2
answers
71k
views
What is the correct way to test for significant differences between coefficients?
I'm hoping someone can help straighten out a point of confusion for me. Say I want to test whether 2 sets of regression coefficients are significantly different from each other, with the following set ...
- 523
21
votes
3
answers
106k
views
Does adding more variables into a multivariable regression change coefficients of existing variables?
Say I have a multivariable (several independent variables) regression that consists of 3 variables. Each of those variables has a given coefficient. If I decide to introduce a 4th variable and rerun ...
- 323
4
votes
2
answers
3k
views
Can I analyze or model a conditional correlation?
In my research I'm looking at the correlation between self-harm and aggression (both continuous). Now, I also have some variables (e.g. depressive symptoms; also continuous) which I do believe ...
- 41
2
votes
1
answer
8k
views
Correction for multiple testing in Multiple regression analysis
Correction for multiple testing (e.g. Bonferroni correction) is recommended when multiple statistical tests are performed on same data. In multiple regression analysis, multiple testing is integral ...
- 9,547
34
votes
4
answers
26k
views
Importance of predictors in multiple regression: Partial $R^2$ vs. standardized coefficients
I am wondering what the exact relationship between partial $R^2$ and coefficients in a linear model is and whether I should use only one or both to illustrate the importance and influence of factors.
...
- 746
16
votes
3
answers
3k
views
Estimating $b_1 x_1+b_2 x_2$ instead of $b_1 x_1+b_2 x_2+b_3x_3$
I have a theoretical economic model which is as follows,
$$ y = a + b_1x_1 + b_2x_2 + b_3x_3 + u $$
So theory says that there are $x_1$, $x_2$ and $x_3$ factors to estimate $y$.
Now I have the real ...
- 457
32
votes
3
answers
48k
views
How to deal with multicollinearity when performing variable selection?
I have a dataset with 9 continuous independent variables. I'm trying to select amongst these variables to fit a model to a single percentage (dependent) variable, ...
- 811
15
votes
1
answer
6k
views
Should I run separate regressions for every community, or can community simply be a controlling variable in an aggregated model?
I am running an OLS model with a continuous asset index variable as the DV. My data is aggregated from three similar communities in close geographic proximity to one another. Despite this, I thought ...
- 309
15
votes
2
answers
4k
views
What is an unbiased estimate of population R-square?
I am interested in getting an unbiased estimate of $R^2$ in a multiple linear regression.
On reflection, I can think of two different values that an unbiased estimate of $R^2$ might be trying to ...
- 44.3k
8
votes
4
answers
4k
views
Why ANOVA/Regression results change when controlling for another variable
This question might be very basic, but somehow I don't understand this point.
Suppose initially I used a univariate regression equation such as
GDP=a+b*Income
...
- 6,284
7
votes
3
answers
7k
views
Why is the intercept in multiple regression changing when including/excluding regressors?
I have a seemingly naive question regarding the interpretation of the intercept in multiple regression. What I found several times is something like this:
The constant/intercept is defined as the ...
- 316
51
votes
6
answers
156k
views
Choosing variables to include in a multiple linear regression model
I am currently working to build a model using a multiple linear regression. After fiddling around with my model, I am unsure how to best determine which variables to keep and which to remove.
My ...
- 1,127
26
votes
1
answer
60k
views
How to deal with high correlation among predictors in multiple regression?
I found a reference in an article that goes like:
According to Tabachnick & Fidell (1996) the independent variables with
a bivariate correlation more than .70 should not be included in
...
- 301
24
votes
3
answers
42k
views
How to split r-squared between predictor variables in multiple regression?
I have just read a paper in which the authors carried out a multiple regression with two predictors. The overall r-squared value was 0.65. They provided a table which split the r-squared between the ...
- 14k