Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
9
votes
1answer
8k views

Assessing the Contribution of each Predictor in Linear Regression

Say I build a linear regression model to identify linear dependencies between variables in my data. Some of these variables are categorical variables. If I want to evaluate the contribution of a ...
8
votes
3answers
24k views

what is the difference between collinearity and interaction?

I cannot differentiate clearly between "interaction" and "collinearity" in multiple linear regression. For me these terms are related but not the same. I have searched the forum but could not find ...
0
votes
1answer
272 views

Principal component (PC) as a substitute for colinear covariates?

I am working on a spatial linear regression and I can tell there is collinearity between covariates. Can I use PCA (Principal Component Analysis) images instead of original covariates to estimate the ...
3
votes
2answers
1k views

Correlation and collinearity in regression

I did a correlation analysis for my variables. All of them are associated (the coefficient is above 0). However, there is no collinearity problem in my regression analysis. I do not know how to ...
2
votes
0answers
749 views

Multivariate analysis for incomplete fractional factorial design

We have conducted a survey experiment with varying amounts of incentive (factor 1 = I1, I2, I3, I4, I5). The experiment was conducted stepwise in three subsequent studies (factor 2 = S1, S2, S3). ...
1
vote
1answer
4k views

Highly correlated predictors in backward stepwise regression?

I know that it's not right to enter variables having multicollinearity (high correlation) into a regression analysis. But if I'm using backward stepwise regression could I add all the highly ...
1
vote
1answer
3k views

Mahalanobis distance in a hierarchical cluster analysis in SPSS

I am conducting a hierarchical cluster analysis in SPSS on my database with several neuropsychological and psychiatric variables. In my database, some of my variables (that is, two pairs of variables) ...
3
votes
1answer
1k views

Time dummies in panel data — absorbing effects?

I am conducting a data analysis. I have a panel with individual firms with firm-specific and macroeconomic variables. I would like to run an OLS regression adjusted for firm clustering effects and ...
1
vote
0answers
107 views

Multicollinearity in a “population model”

I was talking with a colleague who told me that at the time of making logistic regression across a population did not have to worry about assumptions such as multicollinearity, because when analyzing ...
2
votes
3answers
1k views

Pattern mining on a small data set

I have a small data set 30 features/predictors and 30 observations. My target variable is Oil production and my predictors are well & reservoir properties (depth, trajectory, temperature, pressure ...
0
votes
0answers
58 views

Logistic Regression - Two Dependent variables not playing well with each other

My name is Abhi and I trying to teach myself logistic regression by solving some of the problems available on the internet. I am using R and RStudio as the development environment Problem Statement ...
0
votes
1answer
938 views

Low correlation between predictor variables in linear regression

I know that one if one is trying to perform linear regression, multicollinearity can be an issue because it can "lead to unreliable and unstable estimates of regression coefficients." Suppose for a ...
3
votes
0answers
13k views

VIF in GLM model in R

Before running or building a model, ho can we check on the multicollinearity between different covariates in GLM model in R? I know that SAS Proc MIXED procedure gives a column for VIF which is very ...
0
votes
1answer
65 views

Correlated explanatory variables where both are significant

I am running a multiple linear regression using SPSS to test the effect of ethnicity and ethnic/racial attitudes or perceptions on political predispositions. One model, as an example, looks like ...
16
votes
1answer
10k views

Why does Ridge Regression work well in the presence of multicollinearity?

I am learning about ridge regression and know that ridge regression tends to work better in the presence of multicollinearity. I am wondering why this is true? Either an intuitive answer or a ...
1
vote
0answers
60 views

Evidence of collinearity, yet significant coefficients – is this a problem? [duplicate]

I have two main explanatory variables of interest in the model I use. It is only when I include them both that the coefficients are significant. The correlation between them is 0.98 and the vif-value ...
0
votes
1answer
1k views

Multicollinearity with Interaction (high VIF)

When I check the VIF of my independent variables with the dependent variable, it looks normal and less than 5 but when I add the interaction variables, the VIF increase to 48 for some variables. I ...
4
votes
2answers
1k views

Why is least squares performing as well as ridge regression when there is multicollinearity?

I am learning about ridge regression, so I am implementing it in MATLAB as practice. However, I am having trouble finding a structure of data where ridge regression performs better than an ordinary ...
3
votes
1answer
450 views

perfect collinearity among multiple continuous variables

When there is a perfect collinearity among more than two continuous variables, how do you deal with it and how are the regression results interpreted? I have three independent variables which ...
2
votes
1answer
67 views

Non-linear model robustness

I am building a non-linear model aiming to describe the mechanistic process of resource allocation. There several terms, and what makes the model non-linear is competition between lines that are ...
6
votes
1answer
7k views

VIF calculation in regression

I want to use VIF to check the multicollinearity between some ordinal variables and continuous variables. When I put one variable as dependent and the other as independent, the regression gives one ...
18
votes
3answers
49k views

When can we speak of collinearity

In linear models we need to check if a relationship exists among the explanatory variables. If they correlate too much then there is collinearity (i.e., the variables partly explain each other). I am ...
4
votes
1answer
587 views

Is multicollinearity a concern in nonparametric statistics?

Should I be concerned about multicollinearity in nonparametric statistics?
1
vote
1answer
204 views

Can I use simple regression with variables that are not significant in multiple regression?

As I am testing a number of models, I notice that none of my demographic variables are ever signficant. For example, I am testing a model to predict the dependent variable 'perceived substitutability'....
2
votes
0answers
106 views

multicollinearity in OLS regression [duplicate]

If I have a dependent variable Y and two independent variables X1 and X2 , that are highly correlated. Y ~ beta1*X1 + beta2*X2 What issues can multicollinearity cause in an OLS regression, apart ...
3
votes
1answer
2k views

Panel Cointegration, Moderating Effects and Multicollinearity

I am running a panel fixed effects regression on 21 countries and 16 years. Its a secondary data taken from OECD website mostly. My model looks like this: $$ \log{(GDP/Labor)}_{i} = \beta_0 +\beta_1 \...
1
vote
1answer
56 views

Multicollinearity over intervals

You have multicollinearity when you have 2 variables $(X_1, X_2)$ that have a relationship, $X_1=a+X_2$ where $a$ is constant. My question is: is there still a multicollinearity issue if $a$ is not ...
1
vote
0answers
192 views

Is extra sum of squares $SSR(X_p|X_1,…X_{p-1})$ in multiple regression always positive?

An extra sum of squares $SSR(X_p|X_1,...X_{p-1})$, assuming that no pair of predictor variables are perfectly correlated, measures the marginal reduction in the error sum of squares. Eventually one ...
6
votes
3answers
8k views

Is centering a valid solution for multicollinearity?

Let's assume that $y = a + a_1x_1 + a_2x_2 + a_3x_3 + e$ where $x_1$ and $x_2$ both are indexes both range from $0-10$ where $0$ is the minimum and $10$ is the maximum. I found by applying VIF, CI and ...
1
vote
1answer
2k views

PLS Regression and collinearity

From what i know PLS regression is used when there is more variables than observations and when there exist multicollinearity between the independent variables. I have data for a regression model that ...
2
votes
2answers
1k views

Why do both the VIF and tolerance statistics exist, when the latter is just the reciprocal of the former?

Is taking the reciprocal helpful in some way, or is it just a matter of historical accident that there came to be two terms to describe the same thing? (From the Wikipedia page.)
15
votes
2answers
22k views

Variance-Covariance matrix interpretation

Assume we have a linear model Model1 and vcov(Model1) gives the following matrix: ...
1
vote
1answer
454 views

Multicollinearity if independent variables sum to one?

I am intending to explain the variation in dependent variable Y through a number of ratios that, combined, sum to 1 in each period t Hence, $$ Y_t = \alpha + \...
10
votes
1answer
1k views

Variance inflation factor for generalized additive models

In the usual VIF calculation for a linear regression, each independent/explanatory variable $X_j$ is treated as the dependent variable in an ordinary least squares regression. i.e. $$ X_j = \beta_0 + ...
11
votes
2answers
471 views

When including a linear interaction between two continuous predictors, should one generally also include quadratic predictors?

Suppose I am fitting a linear model, and I have two continuous predictors x1 and x2. I think that they might interact, so I add ...
2
votes
1answer
921 views

Dealing with multicollinearity in logistic regresson model

I have 5 quantitative predictor variables in my logit model, and when I use the cor function in R on those 5 variables, I see that $x_{1}$ and $x_{2}$ have ...
3
votes
1answer
1k views

Infer causality with high collinearity

I recently started to ask myself how to measure the impact of education on indexes like GDP: what is the outcome of mathematics or computer science on GDP, at the country level for instance. In this ...
4
votes
0answers
253 views

Techniques for scaling data matrix to avoid rank deficiency issues

I have a $n \times p$ matrix $A$ where $n$ is the number of observables and $p$ is the number of observations. $n \gg p$ In my code, I have done $[E,V] \,=\, eig(A)$ and doing a least squares ...
1
vote
0answers
441 views

How to deal with imperfect multicollinearity

I am estimating a model for an assignment, and found out that there is a 0.9 correlation between two of the independent variables. So If I am not wrong I should omit one variable and redo the ...
4
votes
2answers
4k views

I have my data set, now what?

I have a basic understanding of basic statistics, but I believe I've gotten myself out of my depth. I have a data set with a dependent variable (time span) and three quantitative independent ...
0
votes
1answer
4k views

Collinearity in time dummies, fixed effect regression

I am running a fixed effect panel regression with 81 groups x 20 periods, so approx 1620 (unbalanced) observations. I use the following to create dummies: ...
-3
votes
1answer
3k views

Omitted Variables because of collinearity

...
1
vote
0answers
137 views

Dealing with multicollinearity by smoothing coefficients?

I would like to perform a linear regression. However, the predictor variables are families of variables indexed by time. Let's say the regression problem is: target ~ x(1)+x(2)+...+x(40)+y(1)+...+y(...
5
votes
3answers
1k views

Regression for really small data with high degree of multicollinearity and outliers

I'm working on a promotional response analysis. I have a really small real world dataset with 25 observations and 15 variables. The variables have a high degree of multicollinearity and some have ...
4
votes
0answers
713 views

Multicollinearity in GLM

I have run a general linear model which includes 3 scale independent variables (IVs) and 2 categorical IVs and 1 scale dependent variable. I am testing the assumptions of regression and I a not sure ...
0
votes
1answer
6k views

Confused about multicollinearity, variable selection and interaction terms

I have run a few tests/methods on my data and am getting contradictory results. I have a linear model saying: reg1 = lm(weight = height + age + gender (categorical) + several other variables). If I ...
1
vote
1answer
473 views

Testing for multicollinearity in logistic regression

So far I have checked the tolerance value, VIF and condition indexes. But checking the variance of the regression coefficients I have to wonder: how little variance of the regression coefficient ...
59
votes
3answers
106k views

What is the effect of having correlated predictors in a multiple regression model?

I learned in my linear models class that if two predictors are correlated and both are included in a model, one will be insignificant. For example, assume the size of a house and the number of ...
2
votes
0answers
569 views

Is it possible to center orthogonal polynomials in multiple regression

I have a regression model that looks like the one below. $$ Y = \beta_0 + \beta_1T + \beta_2T^2 + \beta_3T^3 + \beta_4D + \beta_5D*T + \beta_6D*T^2 + \beta_7D*T^3 $$ Where ...
31
votes
5answers
108k views

How to test and avoid multicollinearity in mixed linear model?

I am currently running some mixed effect linear models. I am using the package "lme4" in R. My models take the form: ...