Multicollinearity means predictor variables are correlated with each other, making it harder to determine the role each of the correlated variables is playing. Mathematically, it means the standard errors are increased. Multicollinearity can have counter-intuitive effects.

learn more… | top users | synonyms (1)

0
votes
1answer
23 views

How we can select suitable Variance Influence Factor (VIF) critical value to detect collinearity?

In Variance Influence Factor(VIF) we should use a critical value. A rule of for this value is 10. Is this a good value for detecting collinear based one ...
1
vote
2answers
106 views

How should strongly correlated covariates for logistic regression be treated?

I have to build a multiple logistic regression model with two strongly correlated covariates (predictor variables). How should they be treated? Am I to exclude one of them from the regression? There ...
0
votes
0answers
16 views

two stage? step? regression - passing covariance matrix

I received the following email from one of my colleagues/superiors this morning. I was hoping someone might be able to help me interpret the question and provide a response. I just started this job ...
0
votes
2answers
64 views

Stata automatically tests collinearity for logistic regression?

I'm using Stata for logistic regression. This software automatically checks for collinearity and remove (drop) some variables as we can see below: ...
1
vote
1answer
29 views

Dummy Data & Regression Analysis

I've been doing some analysis projects at work and I've been supplied with some dummy data regarding whether an applicant has applied on a weekday or weekend (I have set this as 0 for weekend and 1 ...
1
vote
0answers
11 views

How to Test Collinearity Between GROUPS of Predictors?

I had a model (made with VW, log loss) based on a set of base (p=1000's) predictors. It did not predict well. I added set A of predictors (p=~5 predictors), and it improved immensely. I added set B ...
1
vote
0answers
21 views

Getting regression coefficients in terms of original variables using PCR

I'm trying to understand Principal Component Regression(PCR) using dummy data (no multicollinearity). I have managed to regress dependent variable on scores but how do I convert the coefficients back ...
0
votes
1answer
53 views

Sign change of a coefficient in logistic regression? [duplicate]

I am running a logistic regression with 5 continuous independent variables (IV). The problem is that IV4 when taken alone has a positive correlation with outcome (coeff > 0), and when taken with the ...
1
vote
2answers
45 views

Multicollinearity and the intercept term with categorial variables

We're given a regression equation with two dummy variables which are perfectly collinear. $$ y_i = \beta_1 D1_i + \beta_2 D2_i + e_i$$ where $ D2_i = 1-D1_i$. Can we estimate this model using least ...
2
votes
3answers
153 views

How does the GLM handle collinear predictors?

In the case of an ordinary least squares GLM with two nearly collinear predictors, how does this shared variance get reflected in the parameter estimates? My understanding is that the parameter ...
2
votes
2answers
38 views

percentages as independent variables and risks of multicollinearity

I am working on a model in which I would have percentages among my independent variables. To be more specific, I would include in the model the variables that, together, would attain a 100 (say I want ...
0
votes
1answer
29 views

Multicollinearity to be assessed item wise or construct wise

In order to check multicollinearity, do I have to combine items for the same construct and assess the correlation table for the constructs, OR do I have to check the item wise correlation table?
1
vote
1answer
79 views

Whether to use height or its z-score or percentile?

I am trying to do regression analysis with level of a chemical in blood as dependent variable and age, gender and weight of children as predicting variables. The sample size is about 5000. Age and ...
2
votes
0answers
56 views

How to systematically remove collinear variables in Python?

Thus far, I have removed collinear variables as part of the data preparation process by looking at correlation tables and eliminating variables that are above a certain threshold. Is there a more ...
0
votes
0answers
44 views

When and when not to use ridge regression

What are the 'indications' (i.e. when to use) and 'contra-indications' (i.e. when not to use) of ridge-regression. I tried to read up on the net and it seems to useful when multi-collinearity is there ...
4
votes
4answers
170 views

How to interpret ridge regression plot

Following is the ridge regression example in MASS package: ...
0
votes
0answers
24 views

Variable selection with hand on example in R

I am looking for most suitable way to perform analysis (with statistical evaluation) where the aim is to find (select) a suit of continuous (collinear) variables that best describe other continuous ...
1
vote
0answers
26 views

Interactions and PCA

I have a linear model that requires a large number of interactions (there are as many interactions as there are IVs) and I want to reduce colinearity using PCA, then regressing the DV on the principal ...
3
votes
3answers
105 views

linear regression - modelling explanatory variables which depend on each other

I'm trying to estimate the value of an apartment, by doing a regression through similar apartments. The regression model looks now like this ...
0
votes
0answers
15 views

Multicollinearity Diagnostics of Mean-Centered Interaction Terms

Interaction terms in moderated regressions exhibit high multicollinearity due to the high correlations with their main effects in case of uncentered data; for Normal distributed variables this is not ...
0
votes
2answers
64 views

Multicollinearity/Redundancy

In a regression problem would it be a problem if I included redundant variables? Ie. Total number of red birds, total number of blue birds, total number of red and blue birds. That would blow up my ...
2
votes
2answers
78 views

Example of interpretation of logistic regression

I was looking at a paper by Pell JS 2009, regarding smoking and survival following acute coronary syndrome. Part of the analysis carried out in the paper was a logistic regression with results as ...
2
votes
2answers
64 views

B is significant correlated with Y, but not a significant predictor of Y in a multiple regression. What can it mean?

I'm working on a paper and I have some problems explaining some of my findings. I have four independent variables, let's call them A,B,C,D. And i have one dependent variable, let's call it Y. In my ...
1
vote
1answer
61 views

Determining Multicollinearity for the given ScatterPlot Matrix

Given this data: should I be concerned with multicollinearity here?
4
votes
1answer
62 views

Is Support Vector Machine sensitive to the correlation between the attributes?

I would like to train an SVM to classify cases (TRUE/FALSE) based on 20 attributes. I know that some of those attributes are highly correlated. Therefore my question is: is SVM sensitive to the ...
1
vote
1answer
33 views

Mathematical Basis behind inflation of Standard errors of Regression estimates due to multicollinearity

We know that due to multi-collinearity, the standard errors of beta estimates get inflated. But what is the mathematical basis to it? I am looking for some mathematical relationship or something to ...
2
votes
0answers
27 views

VIF around 3.5 in two covariates, how shall I deal the problem?

in my multinomial logistic regression model (sample size n=290) I am adjusting the results for a group of covariates (n=8). I tested them for multicollinearity and if most of them have a VIF lower ...
7
votes
2answers
379 views

How to tell the difference between linear and non-linear regression models?

I was reading the following link on non linear regression SAS Non Linear. My understanding from reading the first section "Nonlinear Regression vs. Linear Regression" was that the equation below is ...
1
vote
1answer
94 views

High correlation between two independent variables, but no multicollinearity?

I have two independent variables which have a Pearson correlation coefficient of 0.98. The two independent variables measure the same underlying construct but only at two different points in time ...
0
votes
0answers
36 views

Interactions and Multicollinearity

I have a question concerning multicollinearity in OLS. I built a very simple model: y = a + x(x) + y(x) + e. In this case x is a significant predictor, y is not. Both of them are time related ...
0
votes
0answers
33 views

How can I compute multicollinearity (VIF) in R, and know if it's safe?

I am working on a group project for a university course on "big data research methods". My data is aggregated by neighborhoods in Chicago. My dependent variable $Y$ is the property crime rate (per ...
3
votes
1answer
98 views

Multicollinearity in polynomial regression

How to deal with multicollinearity in polynomial regression? Suppose I have $x$, $x^2$ and $x^3$ as independent variables in my regression equation. How can I calculate and remove multicollinearity ...
0
votes
0answers
32 views

Combining Collinear Variables

I have a set of 10 variables: 9 explanatory, 1 response. I wish to do a constrained regression on the variables and use the values of the coefficients as weights in a TOPSIS analysis. I am having ...
2
votes
2answers
92 views

What is the difference between VIF and stepwise regression?

What is the difference between the variance inflation factor (VIF) and stepwise regression as both help in detecting multicollinearity? What variables are different while running both techniques?
2
votes
1answer
37 views

Automatically fixing ill-conditioning or collinearity

I'm backtesting a regression model, which entails running it on a bunch of bootstrap samples of a "rewound" version of our data set. Unfortunately, in some of these resamplings, I end up getting some ...
0
votes
2answers
66 views

Linear regression multicollinearity

I run linear regression with Posttest scores as DV and Pretest scores and Group as IVs. Collinearity Statistics Tolerance shows .998 both for Pretest and Group (VIF 1.002). Is this one of the ...
0
votes
1answer
54 views

How to Deal with Multicollinearity?

I use cross-sectional macroeconomics variables with OLS. I found that my data suffers from multicolinearity and i am looking for solutions. I read about first differences of the variables and i tried ...
1
vote
0answers
37 views

Multicolinearity in logistic regression using R

Upon performing binary logistic regression, I have found VIF, using R programming, as follows: ...
0
votes
0answers
6 views

How do two variables that with a non-significant low correlation (<.10) become negatively associated in regression? [duplicate]

I have a correlation between variable A and B that is non-significant and small (r = .09). When examining a hierarchical regression with three control variables entered at step 1, and three variables ...
2
votes
0answers
46 views

Multicollinearity and differences between independent variables

I am trying to estimate a model of attrition that takes into account characteristics of an individual's current job and characteristics of an individual's residency. Of particular interest is how the ...
1
vote
0answers
134 views

Multicollinearity Multiple Choice Question

I have one exam question. Anyone can answer and explain the reason for me? You are studying the following $$ Y = \beta_0 +\beta_1X_1 +β_2X_2 +u. $$ You know that the two variables $X_1$ and ...
0
votes
0answers
36 views

Multicollinearity and two variable with the same level of significance

I have a high value of correlation between 2 of my explanatory variables (0.79), and they are both significant at the same level exactly. Besides that they are both important to the model. The ...
1
vote
2answers
229 views

Consequence of Multicollinearity

In case of perfect multicollinearity the predictor matrix is singular and therefore cannot be inverted . Under these circumstances, the ordinary least-squares estimator $\hat\beta=(\Bbb X'\Bbb ...
2
votes
1answer
39 views

VIFs and condition indexes give different answers about multicollinearity

For a multiple regression model, all the variables have p-values below 0.05. The p value for the whole model is below 0.05 as well. When I checked for multicollinearity, I got VIFs below 5 for all the ...
0
votes
0answers
36 views

How does partial least square deal with collinearity?

I have little knowledge about statistics and PLS. Can I answer the question as below? The PLS model is linear, but the coefficients in the model are found in a different way. The principle is that ...
0
votes
0answers
42 views

How to measure the concurvity? (MGCV)

In the R package MGCV there is a function to measure the level of concurvity https://stat.ethz.ch/R-manual/R-patched/library/mgcv/html/concurvity.html What is the formula behind that? Is there any ...
0
votes
1answer
39 views

Multicollinearity in binary logistic regression analysis after latent class analysis

I am attempting to conduct a binary logistic regression but I have come to a roadblock relating to multicollinearity. I originally conducted a latent class analysis which resulted in the ...
0
votes
0answers
55 views

Interpreting signs of coefficient estimates for collinear variables in glmer

I'm doing an glmer on data in which multiple participants attempt to identify multiple words. Word identification accuracy (ACC) is the dependent measure. I want to test whether two variables related ...
4
votes
1answer
303 views

Highly correlated variables in random forest

In my understanding, highly correlated variables won't cause multi-collinearity issues in random forest model (Please correct me if I'm wrong). However, on the other way, if I have too many variables ...
2
votes
6answers
169 views

How to prevent collinearity?

Ieno & Zuur 2015 describe a number of causes of collinearity among explanatory variables entered into a linear regression. One of these causes is what they call a 'data collection' cause. They ...