Multicollinearity means predictor variables are correlated with each other, making it harder to determine the role each of the correlated variables is playing. Mathematically, it means the standard errors are increased. Multicollinearity can have counter-intuitive effects.

learn more… | top users | synonyms (1)

5
votes
1answer
68 views

Why do we use the term multicollinearity, when the vectors representing two variables are never truly collinear?

When two vectors $a$ and $b$ are collinear, then $a = xb$, (where $x$ is a scalar) so in linear algebra, collinearity is a narrowly and clearly defined (and binary) concept. Two vectors -- in my ...
1
vote
0answers
10 views

Variable selection multiple data types

I'm working on a data set containing 45 predictors that cover metric, ordinal and nominal variables. Aiming to finally build a model, which explains the interacting effect of a set of predictor ...
1
vote
1answer
160 views

what if response variable is 'yes or no' in R?

How to analyze above the data to predict the probability that people have disease with a model? Factors thought to influence infection include city, age, and diet. BUT, I don't know how to do ...
2
votes
0answers
46 views

Logistic principal component regression where PCs are correlated with an additional binary predictor

My scenario is this: I collected a bunch of vegetation data (% cover counts in a quadrant at different heights) in patches where birds were seen foraging and also in control patches where no foraging ...
2
votes
1answer
14 views

Covariate related to independent variable - best solutions

I have 2 groups (tinnitus sufferers and controls) who are significantly different in age - I would normally control for age as a covariate (as it is a cognitive task) but it violates the assumption of ...
0
votes
0answers
20 views

Multicollinearity test for panel data

I am using the within estimator (R, plm package) on an unbalanced panel with stock returns as the dependent variable and a number of financial ratios as the independent ones. I want to test for ...
0
votes
1answer
56 views

About stepwise regression and correlation

I am trying to fit a model to some observed data with the least squares method. Now, I am at the stage where I have run a stepwise regression (traditional), with Entry level $=0.025$ and Stay level ...
1
vote
1answer
27 views

Linear regression with redundant features (perfect multicolinearity)

Suppose $X \sim N(0,1)$, $Z=X$, and $Y=X$. An ordinary least squares regression problem is solved: $min_{(b1,b2)} \|Y-(b1*X+b_2*Z)\|_{2}^2$ This is a strictly convex function which must have a ...
1
vote
2answers
61 views

Multicollinearity in multiple regression

I really hope you can help! I'm in the last stages of my PhD. My supervisor is keen on including all variables in the multiple regressions I am running. Some of the scales are intercorrelated (some ...
1
vote
0answers
4 views

Correlation between offset and predictor in count data

I am fitting a Poisson model that requires an offset term to account for sampling effort. However the offset term is highly correlated with one of the linear predictors that is central to my ...
0
votes
0answers
8 views

Coefficients flip sign in general linear model depending on what predictors are included: collinearity is NOT a problem [duplicate]

I have a general linear model with several predictors (~10). The sign (beta) of one of the predictors (Pred1) is negative when all predictors are included. It's STILL negative when the most correlated ...
0
votes
0answers
27 views

Combining collinear continuous predictor variables in GBM

I'm dealing with a dataset (n=254) with one dependent variable (Y), and three independent variables (X1, X2, X3), all continuous. I would like to compare the contribution from each IV to Y. I've been ...
1
vote
0answers
14 views

Centering my variables decreases the condition indices

In my regression model, one of my condition indices is very high, but only the intercept and one of the eplanatory variables have a large variance decomposition proportion for this index. The VIF is ...
2
votes
0answers
45 views

Very low VIF values, but extreme high condition index

In my multiple linear regression model, all of my explanatory variables have a VIF score, lower then 3, but the highest condition index is 709. The constant and one of the explanatory variables have ...
0
votes
0answers
33 views

Regarding analysis of regression result and vif result

I am working on building a regression model. There are 51 points. The number of predictor variables is 37. The following is the result of running lm result. When trying to detecting the ...
0
votes
0answers
20 views

How Can I Build A Regression Model With Collinear Data?

Hello there my fellow Cross Validated members; I’m here today to brainstorm a little bit with all of you out there, to flesh out our collectively acquired data analytic skills, and to try and find new ...
0
votes
0answers
24 views

nonsignificant predictor becomes significant

This may be a question with an obvious answer. I am trying to predict change in a continuous variable over time using a linear mixed model in SPSS. There are 4 time points. Ultimately, I would like to ...
0
votes
0answers
10 views

Substantial changes in significance level when adding more variables to the model [duplicate]

I have a multiple regression model. When I add one more independent variable to the model the significance level of two of my original independent variables suddenly get insignificant. How come? All ...
1
vote
0answers
17 views

Multiple Reg with 2 Independent Variables that are Correlated - Orthogonalizing the IV's

I have two Ind. V's, $x_1$ and $x_2$. They are slightly correlated with eachother. $x_1$ explains a significant portion of $y$'s variability. Rather than just modeling $y = \beta_0 +\beta_1 x_1 ...
0
votes
0answers
27 views

Multicollinearity, feature selection for discriminant analysis and the error rate

I have a question regarding feature selection in LDA/QDA and deciding to eliminate variables to find an optimal model (lowest misclassification rate) I'm looking at how quadratic and linear ...
2
votes
0answers
58 views

Looking for and dealing with collinearity in a GLM

I've got this dataset with one continuous dependent variable and two categorical explanatory variables. I'm wanting to run glms on the data but I'm finding problems with what I think is collinearity. ...
0
votes
0answers
20 views

Mean centering variables NOT in a moderation/interaction term [duplicate]

I am trying to assess the impact of multicollinearity in a regression because I have two separately measured variables which have the reversed signs problem (one predictor is +b regression weight, the ...
1
vote
1answer
76 views

Multicollinearity among categorical variables - Is it normal?

I'm building a logistic regression model in which almost all of the input variables are categorical. There are multiple sets of categorical variables, for example, day of the week, age range buckets, ...
3
votes
3answers
69 views

Too many predictors to manually check linearity

Say I have 1000 potential predictors in a logistic regression. I don't have time to check each predictor manually for linearity. I could wait till after variable selection, but in that case I wonder ...
0
votes
0answers
17 views

Regression on a Ratio with Numerator and Denominator as regressors

Assume we have a dataset with prices of train-rides. There is the price for the ticket, the distance of the ride und some other relevant variables (e.g. x2: 1st/2nd class, x3: name of train-company ...
2
votes
0answers
58 views

Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
0
votes
1answer
43 views

Curse of dimensionality mimics multicollinearity?

Why does the curse of dimensionality mimic multicollinearity, in the following sense.. Consider the random vector $Y = [y_{1}, \dots, y_{4}]$ where each element is ~ Uniform (0,1). Take 10 samples ...
4
votes
2answers
118 views

Many dependent variables, few samples: is this an example of “large $p$, small $n$” problem?

"Large $p$, small $n$" typically refers to "many independent variables, few samples". In my case, I have $1$ independent variable, $300$ dependent variables, and $n < 20$ samples. Thus, my case ...
0
votes
0answers
16 views

Regularization vs PLS for highly colinear data?

When dealing with colinear data, when would I want to use L1/L2 regularization, and when would I want to use Partial Least Squares? Are there any theoretical or practical reasons to prefer one over ...
0
votes
0answers
32 views

vif output for multicolinearity when you have factors

using vif(model) in R provides us with the variance inflation factors for out model. often people use sqrt(vif(model)) > 2 ...
2
votes
1answer
132 views

How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be ...
2
votes
2answers
140 views

Importance of multiple linear regression assumptions when building predictive regression models

As far as I know, one can differentiate between two main goals of the regression analysis: The goal is understanding causal relations between variables. Here, one has to check several common ...
0
votes
0answers
23 views

Multicolinearity: conflicting Spearman rank values and VIF scores

I was wondering if you could provide some advice/feedback on a problem I have been having with detecting multi-colinearity. First, the variables which I am using are not normally distributed, hence I ...
0
votes
1answer
25 views

Why do we use fixed effects and controls at the same time?

When we run a regression model (say OLS for simplicity) of y ~ x, we might have to use several control variables say z1 and z2. Now our model is y ~ x + z1 + z2, we may believe that z1 and z2 are not ...
0
votes
0answers
18 views

Concern about collinearity when adding gender and gender-specific comorbidity for prediction of disease risk

I am build a model to predict the risk of having disease X, let say I have a series of variables and I select the variables to be included in multivariate logistic regression model by: i) clinical ...
2
votes
1answer
165 views

Assessing the Contribution of each Predictor in Linear Regression

Say I build a linear regression model to identify linear dependencies between variables in my data. Some of these variables are categorical variables. If I want to evaluate the contribution of a ...
0
votes
1answer
378 views

what is the difference between collinearity and interaction?

I cannot differentiate clearly between "interaction" and "collinearity" in multiple linear regression. For me these terms are related but not the same. I have searched the forum but could not find ...
0
votes
0answers
42 views

Need to use a percentage as a independent var in regression

This is my first question! My firm takes boxes of documents, preps them (removing staples, taping torn documents), and then scans the documents. I want to calculate what effect staples and torn ...
1
vote
1answer
46 views

Principal component (PC) as a substitute for colinear covariates?

I am working on a spatial linear regression and I can tell there is collinearity between covariates. Can I use PCA (Principal Component Analysis) images instead of original covariates to estimate the ...
2
votes
2answers
97 views

Correlation and collinearity in regression

I did a correlation analysis for my variables. All of them are associated (the coefficient is above 0). However, there is no collinearity problem in my regression analysis. I do not know how to ...
1
vote
0answers
183 views

logistic regression with dummy variables for fractional factorial design

We have conducted a survey experiment with varying amounts of incentive (factor 1 = I1, I2, I3, I4, I5). The experiment was conducted stepwise in three subsequent studies (factor 2 = S1, S2, S3). ...
1
vote
1answer
85 views

Highly correlated predictors in backward stepwise regression?

I know that it's not right to enter variables having multicollinearity (high correlation) into a regression analysis. But if I'm using backward stepwise regression could I add all the highly ...
0
votes
0answers
20 views

Problems with calculation of numerical identification w.r.t. ANOVA smooth for large scale matrices

Suppose we have two (centered) Spline-matrices $\boldsymbol{B_1}$, $\boldsymbol{B_1}$. Then $\boldsymbol{X_1} = [\boldsymbol{B_1},\boldsymbol{B_2}]$ contrains lower order smooths and ...
0
votes
1answer
211 views

Mahalanobis distance in a hierarchical cluster analysis in SPSS

I am conducting a hierarchical cluster analysis in SPSS on my database with several neuropsychological and psychiatric variables. In my database, some of my variables (that is: two pairs of variables) ...
1
vote
1answer
72 views

Time dummies in panel data — absorbing effects?

I am conducting a data analysis. I have a panel with individual firms with firm-specific and macroeconomic variables. I would like to run an OLS regression adjusted for firm clustering effects and ...
0
votes
0answers
56 views

GBM, it's overfitting/multicollinearity problem and parameter setting up

I recently came across a predicting problem (0-1 outcome, with more than 80 variables), I decided to use GBM (Gradient Boosting Machine by Friedman)to handle this job. I let the GBM use only 70% of ...
1
vote
0answers
49 views

Multicollinearity in a “population model”

I was talking with a colleague who told me that at the time of making logistic regression across a population did not have to worry about assumptions such as multicollinearity, because when analyzing ...
2
votes
3answers
310 views

Pattern mining on a small data set

I have a small data set 30 features/predictors and 30 observations. My target variable is Oil production and my predictors are well & reservoir properties (depth, trajectory, temperature, pressure ...
0
votes
0answers
35 views

Logistic Regression - Two Dependent variables not playing well with each other

My name is Abhi and I trying to teach myself logistic regression by solving some of the problems available on the internet. I am using R and RStudio as the development environment Problem Statement ...
0
votes
0answers
88 views

How is predictor importance in a cluster analysis (in SPSS) affected by dichotomy and multicollinearity?

I want to use a cluster analysis (CA) in SPSS to define different profiles in my dataset. I am using different continuous variables for this, including several neuropsychiatric measures. I am new in ...