Multicollinearity means predictor variables are correlated with each other, making it harder to determine the role each of the correlated variables is playing. Mathematically, it means the standard errors are increased. Multicollinearity can have counter-intuitive effects.

learn more… | top users | synonyms (1)

0
votes
0answers
9 views

Looking for and dealing with collinearity in a GLM

I've got this dataset with one continuous dependent variable and two categorical explanatory variables. I'm wanting to run glms on the data but I'm finding problems with what I think is collinearity. ...
0
votes
0answers
20 views

Mean centering variables NOT in a moderation/interaction term [duplicate]

I am trying to assess the impact of multicollinearity in a regression because I have two separately measured variables which have the reversed signs problem (one predictor is +b regression weight, the ...
1
vote
1answer
52 views

Multicollinearity among categorical variables - Is it normal?

I'm building a logistic regression model in which almost all of the input variables are categorical. There are multiple sets of categorical variables, for example, day of the week, age range buckets, ...
3
votes
3answers
63 views

Too many predictors to manually check linearity

Say I have 1000 potential predictors in a logistic regression. I don't have time to check each predictor manually for linearity. I could wait till after variable selection, but in that case I wonder ...
0
votes
0answers
15 views

Regression on a Ratio with Numerator and Denominator as regressors

Assume we have a dataset with prices of train-rides. There is the price for the ticket, the distance of the ride und some other relevant variables (e.g. x2: 1st/2nd class, x3: name of train-company ...
0
votes
0answers
23 views

Is multicollinearity a problem with gradient boosted trees (i.e. GBM)?

A question about multicollinearity for random forests has been asked and answered, but what about boosted trees?
0
votes
1answer
38 views

Curse of dimensionality mimics multicollinearity?

Why does the curse of dimensionality mimic multicollinearity, in the following sense.. Consider the random vector $Y = [y_{1}, \dots, y_{4}]$ where each element is ~ Uniform (0,1). Take 10 samples ...
2
votes
1answer
59 views

large y, small n

"Large p, small n" typically refers to "many independent variables, few samples". In my case, I have 1 independent variable, 300 dependent variables, and < 20 samples. Thus, my case is not the ...
0
votes
0answers
9 views

Regularization vs PLS for highly colinear data?

When dealing with colinear data, when would I want to use L1/L2 regularization, and when would I want to use Partial Least Squares? Are there any theoretical or practical reasons to prefer one over ...
0
votes
0answers
21 views

vif output for multicolinearity when you have factors

using vif(model) in R provides us with the variance inflation factors for out model. often people use sqrt(vif(model)) > 2 ...
2
votes
1answer
75 views

How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be ...
2
votes
2answers
103 views

Importance of multiple linear regression assumptions when building predictive regression models

As far as I know, one can differentiate between two main goals of the regression analysis: The goal is understanding causal relations between variables. Here, one has to check several common ...
0
votes
0answers
14 views

Multicolinearity: conflicting Spearman rank values and VIF scores

I was wondering if you could provide some advice/feedback on a problem I have been having with detecting multi-colinearity. First, the variables which I am using are not normally distributed, hence I ...
0
votes
1answer
18 views

Why do we use fixed effects and controls at the same time?

When we run a regression model (say OLS for simplicity) of y ~ x, we might have to use several control variables say z1 and z2. Now our model is y ~ x + z1 + z2, we may believe that z1 and z2 are not ...
0
votes
0answers
16 views

Concern about collinearity when adding gender and gender-specific comorbidity for prediction of disease risk

I am build a model to predict the risk of having disease X, let say I have a series of variables and I select the variables to be included in multivariate logistic regression model by: i) clinical ...
2
votes
1answer
110 views

Assessing the Contribution of each Predictor in Linear Regression

Say I build a linear regression model to identify linear dependencies between variables in my data. Some of these variables are categorical variables. If I want to evaluate the contribution of a ...
0
votes
1answer
75 views

what is the difference between collinearity and interaction?

I cannot differentiate clearly between "interaction" and "collinearity" in multiple linear regression. For me these terms are related but not the same. I have searched the forum but could not find ...
0
votes
0answers
35 views

Need to use a percentage as a independent var in regression

This is my first question! My firm takes boxes of documents, preps them (removing staples, taping torn documents), and then scans the documents. I want to calculate what effect staples and torn ...
1
vote
1answer
38 views

Principal component (PC) as a substitute for colinear covariates?

I am working on a spatial linear regression and I can tell there is collinearity between covariates. Can I use PCA (Principal Component Analysis) images instead of original covariates to estimate the ...
2
votes
2answers
65 views

Correlation and collinearity in regression

I did a correlation analysis for my variables. All of them are associated (the coefficient is above 0). However, there is no collinearity problem in my regression analysis. I do not know how to ...
1
vote
0answers
104 views

logistic regression with dummy variables for fractional factorial design

We have conducted a survey experiment with varying amounts of incentive (factor 1 = I1, I2, I3, I4, I5). The experiment was conducted stepwise in three subsequent studies (factor 2 = S1, S2, S3). ...
1
vote
1answer
50 views

Highly correlated predictors in backward stepwise regression?

I know that it's not right to enter variables having multicollinearity (high correlation) into a regression analysis. But if I'm using backward stepwise regression could I add all the highly ...
0
votes
0answers
18 views

Problems with calculation of numerical identification w.r.t. ANOVA smooth for large scale matrices

Suppose we have two (centered) Spline-matrices $\boldsymbol{B_1}$, $\boldsymbol{B_1}$. Then $\boldsymbol{X_1} = [\boldsymbol{B_1},\boldsymbol{B_2}]$ contrains lower order smooths and ...
0
votes
1answer
78 views

Mahalanobis distance in a hierarchical cluster analysis in SPSS

I am conducting a hierarchical cluster analysis in SPSS on my database with several neuropsychological and psychiatric variables. In my database, some of my variables (that is: two pairs of variables) ...
1
vote
1answer
36 views

Time dummies in panel data — absorbing effects?

I am conducting a data analysis. I have a panel with individual firms with firm-specific and macroeconomic variables. I would like to run an OLS regression adjusted for firm clustering effects and ...
0
votes
0answers
37 views

GBM, it's overfitting/multicollinearity problem and parameter setting up

I recently came across a predicting problem (0-1 outcome, with more than 80 variables), I decided to use GBM (Gradient Boosting Machine by Friedman)to handle this job. I let the GBM use only 70% of ...
1
vote
0answers
44 views

Multicollinearity in a “population model”

I was talking with a colleague who told me that at the time of making logistic regression across a population did not have to worry about assumptions such as multicollinearity, because when analyzing ...
2
votes
3answers
284 views

Pattern mining on a small data set

I have a small data set 30 features/predictors and 30 observations. My target variable is Oil production and my predictors are well & reservoir properties (depth, trajectory, temperature, pressure ...
0
votes
0answers
34 views

Logistic Regression - Two Dependent variables not playing well with each other

My name is Abhi and I trying to teach myself logistic regression by solving some of the problems available on the internet. I am using R and RStudio as the development environment Problem Statement ...
0
votes
0answers
57 views

How is predictor importance in a cluster analysis (in SPSS) affected by dichotomy and multicollinearity?

I want to use a cluster analysis (CA) in SPSS to define different profiles in my dataset. I am using different continuous variables for this, including several neuropsychiatric measures. I am new in ...
0
votes
1answer
34 views

Low correlation between predictor variables in linear regression

I know that one if one is trying to perform linear regression, multicollinearity can be an issue because it can "lead to unreliable and unstable estimates of regression coefficients." Suppose for a ...
0
votes
0answers
32 views

Multicollinearity

In a regression model with dummy variables, how does one check for intearction between the dummy variable and the independent variables?Wouldn't there be problem of multicollinearity when such ...
0
votes
0answers
141 views

VIF in GLM model in R

Before running or building a model, ho can we check on the multicollinearity between different covariates in GLM model in R? I know that SAS Proc MIXED procedure gives a column for VIF which is very ...
0
votes
1answer
22 views

Correlated explanatory variables where both are significant

I am running a multiple linear regression using SPSS to test the effect of ethnicity and ethnic/racial attitudes or perceptions on political predispositions. One model, as an example, looks like ...
0
votes
0answers
18 views

How to obtain VIF values for independent predictors used in a loglinear poisson generalized linear model?

The correlationmatrix shows that some of my predictors are correlating (Pearson, 0,288 0,492 and 0,360) I think it is useful to have additional information to decide whether this is acceptable or not. ...
2
votes
1answer
56 views

Why does Ridge Regression work well in the presence of multicollinearity?

I am learning about ridge regression and know that ridge regression tends to work better in the presence of multicollinearity. I am wondering why this is true? Either an intuitive answer or a ...
1
vote
0answers
42 views

Evidence of collinearity, yet significant coefficients – is this a problem? [duplicate]

I have two main explanatory variables of interest in the model I use. It is only when I include them both that the coefficients are significant. The correlation between them is 0.98 and the vif-value ...
0
votes
1answer
81 views

Multicollinearity with Interaction (high VIF)

When I check the VIF of my independent variables with the dependent variable, it looks normal and less than 5 but when I add the interaction variables, the VIF increase to 48 for some variables. I ...
2
votes
0answers
72 views

Why is least squares performing as well as ridge regression when there is multicollinearity?

I am learning about ridge regression, so I am implementing it in MATLAB as practice. However, I am having trouble finding a structure of data where ridge regression performs better than an ordinary ...
3
votes
1answer
40 views

perfect collinearity among multiple continuous variables

When there is a perfect collinearity among more than two continuous variables, how do you deal with it and how are the regression results interpreted? I have three independent variables which ...
0
votes
0answers
51 views

Dealing with linear dependent variables

I have a large dataset with many subject each with responses from a consecutive year going back 10 years (ie 100,000 persons per year (not necessarily 10 data points per person as they may not have ...
2
votes
1answer
44 views

Non-linear model robustness

I am building a non-linear model aiming to describe the mechanistic process of resource allocation. There several terms, and what makes the model non-linear is competition between lines that are ...
2
votes
1answer
74 views

VIF calculation in regression

I want to use VIF to check the multicollinearity between some ordinal variables and continuous variables. When I put one variable as dependent and the other as independent, the regression gives one ...
5
votes
3answers
165 views

When can we speak of collinearity

In linear models we need to check if a relationship exists among the explanatory variables. If they correlate too much then there is collinearity (i.e., the variables partly explain each other). I am ...
4
votes
1answer
144 views

Is multicollinearity a concern in nonparametric statistics?

Should I be concerned about multicollinearity in nonparametric statistics?
1
vote
1answer
67 views

Can I use simple regression with variables that are not significant in multiple regression?

As I am testing a number of models, I notice that none of my demographic variables are ever signficant. For example, I am testing a model to predict the dependent variable 'perceived ...
2
votes
0answers
26 views

multicollinearity in OLS regression [duplicate]

If I have a dependent variable Y and two independent variables X1 and X2 , that are highly correlated. Y ~ beta1*X1 + beta2*X2 What issues can multicollinearity cause in an OLS regression, apart ...
3
votes
1answer
134 views

Panel Cointegration, Moderating Effects and Multicollinearity

I am running a panel fixed effects regression on 21 countries and 16 years. Its a secondary data taken from OECD website mostly. My model looks like this: $$ \log{(GDP/Labor)}_{i} = \beta_0 +\beta_1 ...
1
vote
1answer
35 views

Multicollinearity over intervals

You have multicollinearity when you have 2 variables $(X_1, X_2)$ that have a relationship, $X_1=a+X_2$ where $a$ is constant. My question is: is there still a multicollinearity issue if $a$ is not ...
1
vote
0answers
59 views

Is extra sum of squares $SSR(X_p|X_1,…X_{p-1})$ in multiple regression always positive?

An extra sum of squares $SSR(X_p|X_1,...X_{p-1})$, assuming that no pair of predictor variables are perfectly correlated, measures the marginal reduction in the error sum of squares. Eventually one ...