Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

macroeconomic regressors as xreg in ARIMA - differencing required?

I'm forecasting a timeseries that has both trend and seasonality component, which is why I am using ARIMA. Without providing external regressors, the best model selected (in training) has the ...
0
votes
1answer
31 views

Why do the coefficients of cross-sectional fixed effects and time fixed effects become zero?

I currently have a panel data set that contains the quarterly increment of loans initiated from more than 300 cities in China over the period from 2011Q1 to 2020Q2. I want to examine the impact of ...
0
votes
1answer
30 views

How many observations should a dummy variable encompass?

So I feel like this is a rather stupid question, but I can't find a straightforward explanation on the issue. When constructing a dichotomous dummy variable, how many observations should each category ...
3
votes
1answer
74 views

Is it okay to residualize a variable out of my dependent variable, to deal with multicollinearity?

Here is my situation. I have n predictors of interest, and two control variables. If I put them all together in a multiple regression, I get issues with colinearity (i.e., VIFs are very high, and the ...
1
vote
0answers
22 views

Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
7
votes
1answer
137 views

Omitted Variable Bias (OVB) and multicollinearity

In a linear regression model, the reason we control for variables is to prevent the omitted variable bias (OVB). That is, suppose we are trying to fit the model $$ Y = \beta_{0} + \beta_{1}X_{1} + \...
3
votes
1answer
58 views

Getting the wrong sign on a coefficient in logistic regression?

I'm trying to make a logistic regression model explaining whether a law passed last year has affected my dependent variable. My most important variable (an indicator variable for whether the law was ...
1
vote
1answer
13 views

Testing for multi-collinearity after fitting a model with LassoCV from Sklearn in Python?

Is there a way to test for multi-collinearity, like VIF for example, after fitting a model with LassoCV from Sklearn in Python? https://scikit-learn.org/stable/modules/generated/sklearn.linear_model....
1
vote
0answers
20 views

Using a binary variable (present/absent) and its continuous counterpart (amount)

I am working on a banking-related logistic regression problem. Several of my variables are whether someone has an account of some sort, and there is a subsequent variable with the balance of the ...
1
vote
1answer
43 views

Convergence Error Using lifelines.CoxTimeVaryingFitter Python

I want to evaluate my Cox model using the lifelines package for a time varying covariate problem. However, when I use the lifelines.CoxTimeVaryingFitter I get a ...
0
votes
0answers
11 views

Does unbalanced data impact the Variance Inflation Factor?

I am interested in testing whether certain variables with a high correlation between them should be removed from the model. I was thinking of checking this out with VIF. I am working with a data set ...
0
votes
1answer
43 views

Working with systems with Perfect Multicollinearity

I am working with a time-series dataset that is based on demand-supply dynamics with several variables. THe sample data for one time period is: ...
0
votes
0answers
17 views

Working with systems with Perfect Multicollinearity

I am working with a time-series dataset that is based on demand-supply dynamics with variables: Production, Ending Stocks, Exports, Imports etc. I was working on a regression with this data and wanted ...
3
votes
1answer
30 views

Collinearity or not?

In a multiple regression analysis, if you have, in addition to host of other varibales, a dummy i.e. (0,1) coded gender variable with 0=male 1= female, and another dummy variable, say an occupational ...
0
votes
0answers
8 views

Addressing multi-collinearity issues with subset selection methods [duplicate]

I'm a long-time lurker and first-time poster to this forum... I am currently working my way through an Introduction to Statistical Learning, and I have a question regarding the algorithms presented ...
0
votes
0answers
7 views

Variable significance very sensitive to specification of non-correlated second variable

I´m doing research on a political science topic and my models leave me behind with a big questionmark at this point. I have a dataset containing 79 observations on a number of variables and trying and ...
0
votes
0answers
18 views

How does principal component regression help with multicollinearity?

In How to Deal with Multicollinearity?, the top comment in Aaron's answer says "These independent variables are now uncorrelated. Very low eigenvalues also indicate high degrees of ...
0
votes
0answers
32 views

Data with multicollinearity and p >> n

This is a csv file, the file is titled "res_final". The first line contains the names of the variables: ...
0
votes
0answers
9 views

Variance inflation factor vs condition number [duplicate]

We know the Variance inflation factor and condition number both help to measure multicollinearity. Which one should we use when?
2
votes
0answers
32 views

About spurious & masked relationship and multicollinearity

I'm reading Statistical Rethinking by Richard Mclreath and I am bit confused about the subjects in chapter 5. The book itself is about bayesian analysis. This chapter specifically point out to ...
2
votes
1answer
40 views

checking collinearity in a glm

I'm new to checking the VIF value for a glm model so I just want to make sure i"m understanding this correctly. I have 4 predictors for my count model and the model looks like this: ...
0
votes
2answers
34 views

Relationship between variables in a proposed model using linear regression?

I am new to linear regression and I am currently working on a linear regression problem - I have 8 features and one output. The features I am using seem unrelated to each other and I found an article (...
0
votes
0answers
26 views

Best subset regression with instrumental variable

I am applying multiple regression with a data. There are 19 regressors in total and one of them is endogenous. For the endogenous variable I have identified an instrumental variable. When I apply ...
0
votes
0answers
11 views

What is the meaning of the regressor characteristic root?

As described by Greene's Econometric Analysis (7th Edition), the regressor matrix's condition number measures how singular the matrix is. Therefore, the condition number is a measure of ...
0
votes
1answer
47 views

Regression with highly negative correlated variable [duplicate]

I have a regression with 6 independent variables and one of them is highly negatively correlated with one of them. (-0.81 correlation) I was wondering if the regression is still valid or if it's ...
1
vote
0answers
23 views

Testing multi collinearity of ordinal variables in R

I want to test if there is multi collinearity in my dataset. This is made up of multiple ordinal independent variables. Am I correct that this should be done as shown below ...
0
votes
0answers
21 views

How does Ridge regression / Regularization help in selecting less or more important features? [duplicate]

Can someone please explain how regularization helps to shrink the " less important " features to zero ? As far as I know , Regularization only penalizes the weights of ALL the features to ...
0
votes
0answers
17 views

Multicollinearity in ECM model

In Error Correction Model I have proplem related to singular matrix. There are multicollinearity problem of such variables as exchange rate and interest rate. Because they are somehow stable over time,...
2
votes
1answer
35 views

What happens to the coefficients of Ridge and Lasso when you have perfect multicollinnearity?

So let's say we ran a Ridge or Lasso regression on $Y \sim X$, and get coefficient $\beta_X$. Now if we duplicate the $X$, and call it $Z$, and then run the same regression on: $Y \sim X + Z$. How ...
0
votes
0answers
37 views

Does multicollinearity produce wrong beta estimates?

This is the first time for me to ask a question here. I'm sorry that if I break any rule here. I have encountered a problem about the consequence of multicollinearity. During reading the explanation ...
1
vote
1answer
41 views

VIF scores for ordinal independent variables

I suspected there was a high degree of multicollinearity in the independent variables of my data. Each of these variables is ordinal. The original model is ...
4
votes
1answer
59 views

Testing multi-collinearity of ordinal independent variables in R

I am trying to conduct an ordinal logistic regression, but I first want to test if I fulfill the assumption of no multicollinearity. All of my 8 independent variables are ordinal with up to 5 levels. ...
4
votes
1answer
61 views

What is the difference between a confounder, collinearity, and interaction term?

These terms kind of confuse me because they all seem to imply a certain correlation. Confounder: influences dependent and independent variable Collinearity: to me just means correlation between ...
0
votes
0answers
18 views

Condition numbers, invertibility and multicollinearity

The following is an excerpt from Greene's Regression Analysis (Seventh Edition): a) What does it mean to be "difficult" to invert a matrix accurately? Shouldn't all matrixes be either ...
2
votes
1answer
24 views

Nested ANOVA: Correct use for colinearity problem?

We have neuronal data, that we want to analyse for effects of two factors that are hierarchically structured. The data: We have recorded neuronal data from a neuron, where the organism received visual ...
1
vote
0answers
32 views

Why does centering NOT cure multicollinearity?

In several posts, such as Is centering a valid solution for multicollinearity?, it states that centering doesn't solve multicollinearity because "it's a linear transformation." I just made ...
1
vote
0answers
17 views

Multicollinearity negating $\beta$

In Regression coefficients that flip sign after including other predictors, ars's answer states that "Basically, if your variables are positively correlated, then the coefficients will be ...
4
votes
2answers
77 views

How to identify which variables are collinear in a singular regression matrix? [duplicate]

I have a matrix on which I am performing a zero inflated regression model but it return an error indicating collinearity between some of the variables ...
0
votes
1answer
23 views

Should we remove a variable having low p value but high multicollinearity?

I have two regression models. The 2nd model is obtained by removing one variable from the first. The removed variable had high multicollinearity although very low p-value. (A variable RGDP was showing ...
0
votes
2answers
59 views

How does colinearity among the features impact classification problem in a) Logistic Regression b) Tree models like Decision Tree or Random Forest? [duplicate]

If I have 20 variables, should I do a pair-wise correlation check before building a classification model with Logistic Reg, Decision Tree or Random Forest ? Multicolinearity is a problem in regression....
4
votes
0answers
51 views

multicollinearity and predictive power

In wikipedia it says ...
1
vote
1answer
34 views

Geometric interpretation of multiple regression

I have two predictors $X_1$ and $X_2$, they are positively correlated. The response $Y$ are positively corrected with both $X_1$ and $X_2$, i.e. if we fit regression $Y=\beta_1 X_1$ and $Y=\beta_2 X_2$...
1
vote
1answer
42 views

why standardizing the variables lessen multicollinearity in linear regression

I've seen that by standardizing variables with subtracting their means, the VIF drops significantly below threshold of 5. But originally they were >10. What's the mathematical proves that ...
0
votes
0answers
5 views

Does changing reference variable of design matrix reduce multicollinearity?

It is stated here by Paul Allison that, if multicollinearity occurs within the design matrix of a categorical variable, multicollinearity can be reduced by setting the category with higher share of ...
4
votes
1answer
44 views

Collinearity in linear mixed models

Is collinearity of the independent fixed effects (covariates, so not the variables I'm interested in) a problem when you create a linear mixed model? Is there a good resource I can point to to support ...
1
vote
2answers
55 views

What's the difference between endogenous variable and multicollinearity?

Investopedia says that: Multicollinearity is a statistical concept where independent variables in a model are correlated. Also Investopedia says that: An endogenous variable is a variable in a ...
2
votes
0answers
19 views

Multicollinearity and Partial Dependence Questions

Assume I build a binary classification model to predict p(y=1) from {x1, x2, ... x10} For now, assume that model could be a GBM, RandomForest, or Logistic Regression. Also assume that all of the ...
0
votes
0answers
29 views

Working around multicollinearity

Given that I am far from an expert statistician, perhaps the community can inform me if my first thought to work around multicollinearity is going to result in questionable results. I have a model ...
0
votes
1answer
42 views

Variance Inflation Factors for a glm with clustered standard errors

I am using the glm.cluster function in R package miceadds and I would like to calculate the variance inflation factors (VIF), much as the vif function in R package car does. If I try to use car::vif I ...
0
votes
0answers
11 views

Multiple targets in dataset - feature selection

I have a dataset with several features and three KPIs to be seen as individual targets. The question I have is the following: Can I use for example the 2nd and 3rd target as a predictor for the model ...

1
2 3 4 5
19