Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. ...

1
vote
1answer
32 views

How can I use linear/logistic regression for inference with colinear variables and a smallish dataset?

I have a dataset of around 120 observations, with 30 calculated variables and I am trying to predict a continuous response (result of an experiment) using those 30 variables. Ideally the smallest ...
0
votes
0answers
10 views

Steps after calculating VIF to deal with multi collinearity

How do we eliminate variables after calculating VIF ? I have read about step wise removal of variables with high VIF till we reach VIF values below 5. On the other hand , my professor told me that ...
0
votes
0answers
12 views

Transforming panel data OLS into cross-sectional data model

I am currently stuck on a task where I am interested in estimating the production function for agricultural output using panel data as follows: \begin{equation} y_{it} = x_{it}\beta + \alpha_i + \...
0
votes
0answers
18 views

Consistent estimate of $\beta$ in linear regression under multicollinearity

I am currently stuck on a task where I am interested in estimating the production function for agricultural output as follows: \begin{equation} y_{i} = x_{i}\beta + \alpha_i + \epsilon_{i} \end{...
0
votes
0answers
24 views

How Can I Test for Multicollinearity in my Categorical Predictor Variables when Doing Ordered Logistic Regression?

I have a dataset where my dependent variable measures 'How much Trump’s locker room video should have mattered in the election'. The categories are coded between 1-5, where 1 represents 'should not ...
0
votes
0answers
20 views

How multicollinearity affects out-of-sample performance

I want to understand the impact of multicollinearity to the predictive power of a linear model. So I doodled some math. Say we have a model $y=X\beta+\epsilon$. Fix a test data $x_{test}$ but let the ...
0
votes
0answers
9 views

collionarity diagonistics

from sas output I got two tables for collionarity diagnostics. Collinearity Diagnostics and Collinearity Diagnostics with intercept adjusted. Which table should I interpret for eigen values eigen ...
0
votes
0answers
6 views

Is it sufficient to tackle collinearity by correlation matrix? [duplicate]

I am doing a regression project with my groupmates. The response variable is house price and the regressors are mixture of numeric, ordinal and categorical variables. We come upon collinearity issue. ...
2
votes
1answer
95 views

Does multicollinearity cause type I errors?

Citing Wikipedia on multicollinearity, One of the features of multicollinearity is that the standard errors of the affected coefficients tend to be large. In that case, the test of the hypothesis ...
0
votes
1answer
15 views

evaluate relationship between 1 independent variable and multiple dependent variables.

To evaluate the relationship between variables coefficient of correlation and coefficient of determination obtained from regression are used. But what is the procedure if I have 1 Independent variable ...
2
votes
0answers
19 views

Concurvity in additive modelling. Parametric term?

I am modelling a outcome in gam were two variables (x1 and x2) are continuous and the other four are factors. I have a suspicion that x1 and x2 might be collinear so I want to check that. As I ...
0
votes
0answers
23 views

Handling collinearity in GAM

I am interesting if a variable x1 is of importance for the outcome y. To investigate this I am trying to fit a model A1 <- gam(y~ s(x1) + s(x2) + x3 + x4 + x5 + x6)) where x1 and x2 are ...
0
votes
0answers
22 views

Multicollinearity - doubt on regression coefficients [duplicate]

Why is it the case that when there is multicollinearity, the regression coefficients become indeterminate and their standard errors become infinite?
1
vote
1answer
27 views

Is there a collinearity issue when using: x, dummy indicating extreme negative value of x and their interaction?

I was wondering whether I can build my baseline model using the following variables without incurring in any multicollinearity issue: $X_1$= Net capital flows over GDP (which may be positive and ...
1
vote
0answers
21 views

VIF - Variance Inflation, when to remove the variable

I'm doing a regression analysis on cement mixtures. The goal is obviously to create the mixture with the most strength. Here are the following variables for me to work with: Variables: Strength = ...
0
votes
1answer
28 views

What does “regression of predictor onto all of the other predictors” mean?

I encountered a lot of references that talk about R squared but I can't understand what the difference is between the R squared in regression of the response on the predictors and the R squared that ...
0
votes
0answers
15 views

Colinearity and Variance Inflation Factor

I want to ask a question regarding colinearity and Variance Inflation Factor (VIF). I started with 7 variables, and have excluded one since there were not correlation between that predictor variable ...
0
votes
0answers
13 views

Including transformations of the same predictor in a multiple linear regression - inflated variances?

I'm working through Introduction to Statistical Learning and in chapter 3 (Linear Regression), I learn that if a relationship between predictors and a response is not linear, I may use cubic ...
0
votes
0answers
15 views

“Collinear” categories “nested” within another in multivariate analysis

Apologies for the title, but did not know how to describe this scenario that I can't wrap my head around. It is not the scenario of my analysis, but I'm describing an analogous situation. Study ...
2
votes
1answer
50 views

Statistical significance testing for multiple categorical variables: modeling coefficient of variation of temperature inside houses

I am a beginner in R (and stats), so please excuse the simple nature of my question. I have a range of variables all relating to the physical and social characteristics of households in the UK. I am ...
2
votes
0answers
37 views

Is multicolinearity testing altogether useless if the p-value on each regressor is less than 0.01?

Isn't the effect of multicolinearity to artificially increase the standard errors and thus artificially decrease the t-statistics, and thus artificially increase the p-values? If so, if all ...
1
vote
1answer
32 views

How to use Design of Experiment (DoE) to reduce the number of simulations?

I am planning to do simulation for parametric study and there are 9 parameters in total. I was suggested to use DoE to reduce the number of simulations that I need to do. I studied the basic of DoE ...
0
votes
0answers
81 views

Inference from regression in presence of multicolinearity

I would like to estimate the effect of one independent variable on the predicted variable in the purely observational study. On the other hand I know that there exists another independent variable, ...
0
votes
0answers
15 views

Are these two variables collinear in regression?

I have a dummy variable equal to 1 every year of road access, and another dummy variable equal to 1 in the year connection started. Can I include these two variables together as explanatory variables?...
1
vote
0answers
15 views

Exploratory factor analysis of a non-normally distributed data set with probable multicollinearity problems

I am trying to develop a scale measuring employee satisfaction regarding organizational support using spss. Most of my variables are positively skewed which should not be a problem in EFA as long as ...
1
vote
1answer
27 views

If x4 has high negative correlation with x1 - x3, which all correlate highly with each other, will x4 also be difficult to estimate in a regression?

Say you have predictor variables x1 through x4 and response y. x1, x2, x3 all highly correlate with each other, thus they will have inflated standard errors due to that multicollinearity. Meanwhile x4 ...
1
vote
1answer
41 views

What to do if recombination of independent variables cause multicollinearity issue?

Let's say you use a regression that has either: 1) interaction variables or 2) polynomials. When using those features you may run into multicollinearity issues. Do you know how to resolve this issue?...
1
vote
4answers
68 views

Can we use covariance matrix to examine feature collinearity?

Consider using Multi-variate Gaussian to approximate $X = [X_1, X_2, ..., X_n]$ and $X_i = [x_{i1}, x_{i2}, x_{i3}, ..., x_{im}]$, so we have n data points and each data point has m features. Multi-...
1
vote
0answers
23 views

What regression diagnostics should I perform for an ordered probit?

Currently I have done the following diagnostics with the linktest multicollinearity with vif the parallel lines assumption with lr test of the oprobit and goprobit. I have seen that I may have to ...
0
votes
2answers
45 views

ANOVA multicollinearity adjustment

I am using the statsmodel.ols module to compute an omnibus (ANOVA) F-test for three within-subjects factors; 2*3*2 levels design. The Cond. No. of the omnibus test (...
1
vote
2answers
66 views

Linear model with two columns where one column is a linear transformation of another

If we have a linear model $y_t = β_1x_{1t} + β_2x_{2t} + e_t$ where the errors $e_t$ are i.i.d normally distributed: $e_t$ ~ $IIN(0, σ^2)$, t = 1,...,T. The vectors of random regressors $X_1$ and $...
4
votes
1answer
54 views

VIF Drops Significantly When I Delete Some Dummy Variables

Is my model valid even with the high VIF? Does it matter which dummy variable I drop as the reference point? I have a a category variable (Fruit) that I converted ...
0
votes
2answers
97 views

Logistic Regression: multicollinearity and Kappa statistics

I may be wrong but from my understanding logistic regression requires there to be little or no multicollinearity among the independent variables, and yet Kappa statistics as part of postResample() ...
1
vote
2answers
66 views

multicollinearity resulting in high variance

Section 8.7.1 of Elements of Statistical Learning talks about high variance in a classification tree due to high correlation between features. What is the intuition behind this? I would think that ...
0
votes
1answer
69 views

checking multicollinearity with generalized additive model in R

I am trying to check multicollinearity with GAM using VIF in R. Should I use vif from the package car ? or Is it right way to check vif using vif.gam from package mgcv::helper:: ? It gives quite ...
1
vote
0answers
23 views

VECM with Multicollinearity

I have fit a vector error correction model (VECM) to some macroeconomic data. In particular, I am interested in three relationships real GDP as a function of employment and real wages employment as a ...
4
votes
2answers
73 views

Practical limits to collinearity problems?

Collinear independent variables can have undesirable effects on the interpretation of coefficients in a linear model. Indeed, for two perfectly-correlated predictors, the coefficients are not ...
1
vote
0answers
59 views

How to use Elastic Net Model to Reduce Collinearity

I am using R to perform a linear regression with a dataset that has clearly correlated independent variables (collinearity). I am using the vif (variance inflation factor) function from the car ...
2
votes
0answers
46 views

Elastic Net and collinearity

I am performing elastic net for variable selection on a dataset of 95 records and 41 variables. The response is a continuous numerical. I choose the alpha and lambda parameters through 10 fold cross ...
0
votes
0answers
54 views

Multicollinearity in Linear Mixed Models

I am currently working with longitudinal data and want to use linear mixed models (LMM) to perform analysis. However, I want to investigate the issue of multicollinearity in the data. But this got me ...
0
votes
0answers
82 views

VIF giving “Inf” values in R

I have a dataset(df) of 84x186 variables (after removing dependent variable). I am trying to solve a classification problem here. I used simple cor(df) and found ...
1
vote
1answer
22 views

Multicollinearity in BLR

Please I am exploring for multicollinearity in my data between socio-economic characteristics. I ran a collinearity diagnostic test and I have a conditional index of 6.235 which is less than 10. I ...
0
votes
1answer
43 views

Multicollinearity [closed]

Please I am exploring for multicollinearity in my data between socio-economic characteristics. I ran a collinearity diagnostic test and I have a conditional index of 6.235 which is less than 10. I ...
1
vote
1answer
53 views

Multicollinearity confusion

so for my master's thesis, I am examining the influence of union density (% of the workforce in a union) and top marginal tax rates on pre-tax CEO pay. These two independent variables are very highly ...
0
votes
1answer
130 views

Can PCA be used for detecting multicollinearity?

The definition of multicollinearity is: Given a set of $N \times 1$ predictors $X = (x_1, x_2, \cdots, x_m)$, if $$x_j = \sum_{i \neq j}a_ix_i$$ then we say there is multicollinearity among the ...
0
votes
0answers
37 views

multi-collinearity and random forest [duplicate]

I understand that the general linear model will fail when data exist multicollinearity. Does random forest suffer from multicollinearity? Is there any published reference studying about this issue?
2
votes
0answers
24 views

What is the difference between multicollinearity and correlation?

I have a set of data and I wish to construct a multivariate regression model for predicting. I saw that if the variables are multi-colinear the multivariate regression model will be bad. I don't know ...
2
votes
0answers
14 views

How to Interpret Co-Linear Interaction Term?

I am looking to model transit ridership based on demographics at the stop level. For each stop, I know: ONS: Total boardings per day - averaged over the number of times a bus pulls up at the stop ...
0
votes
1answer
38 views

Checking for multicollinearity in selection of variables for regression model

For the selection of variables for a regression model, I did a pairwise correlation matrix between the different predictors and the response variable. From the pairwise correlation matrix, I realise ...
0
votes
2answers
40 views

multi-collinearity in a time series?

Say I have a set of time series data spanning 2000-2016 I code my years as the variable time, starting in 2000 as 0, 1, 2,....15 Say I want to compare the bush presidency to the obama presidency and ...