Multicollinearity means predictor variables are correlated with each other, making it harder to determine the role each of the correlated variables is playing. Mathematically, it means the standard errors are increased. Multicollinearity can have counter-intuitive effects.

learn more… | top users | synonyms (1)

0
votes
0answers
46 views

Multicollinearity classification regression with R [closed]

I want to do multiple regression, but my explanatory variables are highly correlated with each other. Some are roughly a linear combination of others. To deal with that, I tried to analyse my data ...
0
votes
1answer
19 views

Correlated explanatory variables where both are significant

I am running a multiple linear regression using SPSS to test the effect of ethnicity and ethnic/racial attitudes or perceptions on political predispositions. One model, as an example, looks like ...
0
votes
0answers
11 views

How to obtain VIF values for independent predictors used in a loglinear poisson generalized linear model?

The correlationmatrix shows that some of my predictors are correlating (Pearson, 0,288 0,492 and 0,360) I think it is useful to have additional information to decide whether this is acceptable or not. ...
2
votes
1answer
45 views

Why does Ridge Regression work well in the presence of multicollinearity?

I am learning about ridge regression and know that ridge regression tends to work better in the presence of multicollinearity. I am wondering why this is true? Either an intuitive answer or a ...
1
vote
0answers
42 views

Evidence of collinearity, yet significant coefficients – is this a problem? [duplicate]

I have two main explanatory variables of interest in the model I use. It is only when I include them both that the coefficients are significant. The correlation between them is 0.98 and the vif-value ...
0
votes
1answer
54 views

Multicollinearity with Interaction (high VIF)

When I check the VIF of my independent variables with the dependent variable, it looks normal and less than 5 but when I add the interaction variables, the VIF increase to 48 for some variables. I ...
2
votes
0answers
47 views

Why is least squares performing as well as ridge regression when there is multicollinearity?

I am learning about ridge regression, so I am implementing it in MATLAB as practice. However, I am having trouble finding a structure of data where ridge regression performs better than an ordinary ...
3
votes
1answer
31 views

perfect collinearity among multiple continuous variables

When there is a perfect collinearity among more than two continuous variables, how do you deal with it and how are the regression results interpreted? I have three independent variables which ...
0
votes
0answers
48 views

Dealing with linear dependent variables

I have a large dataset with many subject each with responses from a consecutive year going back 10 years (ie 100,000 persons per year (not necessarily 10 data points per person as they may not have ...
2
votes
1answer
39 views

Non-linear model robustness

I am building a non-linear model aiming to describe the mechanistic process of resource allocation. There several terms, and what makes the model non-linear is competition between lines that are ...
4
votes
3answers
109 views

When can we speak of collinearity

In linear models we need to check if a relationship exists among the explanatory variables. If they correlate too much then there is collinearity (i.e., the variables partly explain each other). I am ...
4
votes
1answer
131 views

Is multicollinearity a concern in nonparametric statistics?

Should I be concerned about multicollinearity in nonparametric statistics?
1
vote
1answer
41 views

Can I use simple regression with variables that are not significant in multiple regression?

As I am testing a number of models, I notice that none of my demographic variables are ever signficant. For example, I am testing a model to predict the dependent variable 'perceived ...
2
votes
0answers
25 views

multicollinearity in OLS regression [duplicate]

If I have a dependent variable Y and two independent variables X1 and X2 , that are highly correlated. Y ~ beta1*X1 + beta2*X2 What issues can multicollinearity cause in an OLS regression, apart ...
3
votes
1answer
75 views

Panel Cointegration, Moderating Effects and Multicollinearity

I am running a panel fixed effects regression on 21 countries and 16 years. Its a secondary data taken from OECD website mostly. My model looks like this: $$ \log{(GDP/Labor)}_{i} = \beta_0 +\beta_1 ...
0
votes
1answer
31 views

Multicollinearity over intervals

You have multicollinearity when you have 2 variables $(X_1, X_2)$ that have a relationship, $X_1=a+X_2$ where $a$ is constant. My question is: is there still a multicollinearity issue if $a$ is not ...
1
vote
0answers
43 views

Is extra sum of squares $SSR(X_p|X_1,…X_{p-1})$ in multiple regression always positive?

An extra sum of squares $SSR(X_p|X_1,...X_{p-1})$, assuming that no pair of predictor variables are perfectly correlated, measures the marginal reduction in the error sum of squares. Eventually one ...
3
votes
3answers
317 views

Is centering a valid solution for multicollinearity?

Let's assume that $y = a + a_1x_1 + a_2x_2 + a_3x_3 + e$ where $x_1$ and $x_2$ both are indexes both range from $0-10$ where $0$ is the minimum and $10$ is the maximum. I found by applying VIF, CI and ...
0
votes
1answer
36 views

PLS Regression and collinearity

From what i know PLS regression is used when there is more variables than observations and when there exist multicollinearity between the independent variables. I have data for a regression model that ...
0
votes
0answers
34 views

How to deal with multicollinearity without dropping variables

In establishing a linear model, if two variables are highly correlated, then it turns out to have high VIFs. If we applied data reduction techniques, like PCA, we will lose explainability to some ...
0
votes
0answers
42 views

Identifiability in linear regression

If we have a generative model: $X_2=X_1a_1+\varepsilon$, where $\varepsilon \sim \mathcal{N}(0,\sigma_2^2)$, do we have $X_1=X_2a_2+\varepsilon '$, where $\varepsilon \sim \mathcal{N}(0,\sigma_1^2)$ ...
2
votes
2answers
65 views

Why do both the VIF and tolerance statistics exist, when the latter is just the reciprocal of the former?

Is taking the reciprocal helpful in some way, or is it just a matter of historical accident that there came to be two terms to describe the same thing? (From the Wikipedia page.)
6
votes
2answers
360 views

Variance-Covariance matrix interpretation

Assume we have a linear model Model1 and vcov(Model1) gives the following matrix: ...
1
vote
1answer
53 views

Multicollinearity if independent variables sum to one?

I am intending to explain the variation in dependent variable Y through a number of ratios that, combined, sum to 1 in each period t Hence, $$ Y_t = \alpha + ...
4
votes
0answers
73 views

Variance inflation factor for generalized additive models

In the usual VIF calculation for a linear regression, each independent/explanatory variable $X_j$ is treated as the dependent variable in an ordinary least squares regression. i.e. $$ X_j = \beta_0 + ...
3
votes
0answers
58 views

When including a linear interaction between two continuous predictors, should one generally also include quadratic predictors?

Suppose I am fitting a linear model, and I have two continuous predictors x1 and x2. I think that they might interact, so I add ...
1
vote
1answer
73 views

Dealing with multicollinearity in logistic regresson model

I have 5 quantitative predictor variables in my logit model, and when I use the cor function in R on those 5 variables, I see that $x_{1}$ and $x_{2}$ have ...
1
vote
0answers
62 views

Infer causality with high collinearity

I recently started to ask myself how to measure the impact of education on indexes like GDP: what is the outcome of mathematics or computer science on GDP, at the country level for instance. In this ...
0
votes
0answers
25 views

handling multicollinearity by backwards regression and omitted variable bias

Suppose I try to estimate a production function as follows: $logY=b_0+b1*logX_1+b_2*logX_2+b_{11}*(logX_1)^2+b_{22}*(logX_2)^2+b_{12}*(logX_1)*(logX_2)+u$, where $Y$ is the output, $X_1$, $X_2$ are ...
4
votes
0answers
42 views

Techniques for scaling data matrix to avoid rank deficiency issues

I have a $n \times p$ matrix $A$ where $n$ is the number of observables and $p$ is the number of observations. $n \gg p$ In my code, I have done $[E,V] \,=\, eig(A)$ and doing a least squares ...
1
vote
0answers
32 views

How to deal with imperfect multicollinearity

I am estimating a model for an assignment, and found out that there is a 0.9 correlation between two of the independent variables. So If I am not wrong I should omit one variable and redo the ...
4
votes
2answers
199 views

I have my data set, now what?

I have a basic understanding of basic statistics, but I believe I've gotten myself out of my depth. I have a data set with a dependent variable (time span) and three quantitative independent ...
0
votes
1answer
208 views

Collinearity in time dummies, fixed effect regression

I am running a fixed effect panel regression with 81 groups x 20 periods, so approx 1620 (unbalanced) observations. I use the following to create dummies: ...
1
vote
0answers
52 views

Dealing with multicollinearity by smoothing coefficients?

I would like to perform a linear regression. However, the predictor variables are families of variables indexed by time. Let's say the regression problem is: target ~ ...
4
votes
3answers
284 views

Regression for really small data with high degree of multicollinearity and outliers

I'm working on a promotional response analysis. I have a really small real world dataset with 25 observations and 15 variables. The variables have a high degree of multicollinearity and some have ...
1
vote
0answers
74 views

Multicollinearity in GLM

I have run a general linear model which includes 3 scale IVs and 2 categorical IVs and 1 scale DV. I am testing the assumptions of regression and I a not sure how to test multicollinearity since I ...
0
votes
1answer
204 views

Confused about multicollinearity, variable selection and interaction terms

I have run a few tests/methods on my data and am getting contradictory results. I have a linear model saying: reg1 = lm(weight = height + age + gender (categorical) + several other variables). If I ...
1
vote
1answer
71 views

Testing for multicollinearity in logistic regression

So far I have checked the tolerance value, VIF and condition indexes. But checking the variance of the regression coefficients I have to wonder: how little variance of the regression coefficient ...
0
votes
0answers
34 views

Utilizing PCA when rotated factor loadings are difficult to relate to real world variables?

In my ongoing attempts to make sense of a highly correlated dataset without just throwing out most of the variables i am at the point of attempting to interpret PCA output. I hope i am right in the ...
6
votes
3answers
201 views

What is the effect of having correlated predictors in a multiple regression model?

I learned in my linear models class that if two predictors are correlated and both are included in a model, one will be insignificant. For example, assume the size of a house and the number of ...
0
votes
0answers
20 views

PCA, non positive definitive & potential cures?

I am trying to use PCA to remove the correlation in my dataset My correlation matrix is being reported as non positive definitive As I understand it a NPD report means that my transposed matrix does ...
0
votes
0answers
20 views

Correlated varaibles and bootstrap forest possible problem?

I have been using this method to reduce the number of variables for a model but i have just had a bad thought. Due to the nature of bootstrap is it possible that by randomly using variables you are ...
1
vote
0answers
78 views

Is it possible to center orthogonal polynomials in multiple regression

I have a regression model that looks like the one below. $$ Y = \beta_0 + \beta_1T + \beta_2T^2 + \beta_3T^3 + \beta_4D + \beta_5D*T + \beta_6D*T^2 + \beta_7D*T^3 $$ Where ...
2
votes
0answers
705 views

How to test and avoid multicollinearity in mixed linear model?

I am currently running some mixed effect linear models. I am using the package "lme4" in R. My models take the form: ...
1
vote
0answers
69 views

Multicollinearity of Socioeconomic index vs. wealth variables separately?

I am running Cox regressions in a large dataset to determine effects of individual and area characteristics on mortality outcomes. (My model is not a multilevel model, as the initial significant ...
0
votes
1answer
73 views

Collinearity among explanatory variables

My correlation matrix is not showing very high correlations (less than 0.5), but still, when I am regressing panel fixed effects models, all my variables are omitted, and it shows it is due to ...
1
vote
2answers
130 views

Multicollinearity when adding a confounding variable

When you run a regression on ice cream sales with predictor shark attacks, you find a significant coefficient. But that is because there is a confounding variable temperature. But how do you correct ...
0
votes
0answers
35 views

Confirming multicollinearity with small number predictors (after initial model construction)

I am working on a dataset with with 100 or so X variables and the effects on a single Y. As with previous threads this contains a lot of fairly significant correlations. I considered using PCA but ...
0
votes
1answer
48 views

Fitting same classes of many many collinear variables

I am trying to build a linear model of the simple form, y = a + b1*x1 + b2*x2 + b3*x3 + c1*z1 + c2*z2 + c3*z3.... The characteristics of the model are 1) Each of the independent variable belongs to ...