Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
0
votes
1answer
26 views

scikit-learn linear regressor digests perfectly collinear features?

I am currently running this little piece of code: ...
0
votes
0answers
30 views

Index Breakdown

Suppose I created an index, $I$, such that: $$I = aX + bY + cZ$$ If I regressed I against another variable such that: $$B = dI + eA$$ Would rewriting $B$ as $$B= d(aX + bY + cZ) + eA$$ allow us to ...
1
vote
0answers
63 views

Creating a correlation matrix for categorical variables in R

I am new to R and have been trying to solve this problem by myself for many hours without success. I am conducting logistic regression and have used the glm() function for univariable analysis. I have ...
0
votes
1answer
19 views

Issues in testing a simple linear relationship. Collinearity? Misspecification? Any other insight?

I have a theoretical model saying that Y should be equal to: Y = X + c * (W - X) + (Z1 - Z2), where c is a given constant. Here, it may be important to say that X is measured with error. Someone ...
2
votes
1answer
44 views

How to deal with predictors which are not significant, although r-squared is significant?

I did factor analysis and found three factors. To examine if which factors significantly affected a certain dependent variable, I added all three factors to a regression model. The correlation ...
1
vote
1answer
44 views

How does PLSR solve Multicollinearity

We know that PLSR is a very common way to solve Multicollinearity in the Multiple Linear Regression. But do you know how does it work in detail? And why Multicollinearity of $x$ will be related to the ...
0
votes
0answers
17 views

contradiction between PCA and Multicollinearity

From here Should one remove highly correlated variables before doing PCA? we know that when there are some highly correlated Features during the PCA, we should remove them to avoid some incorrect ...
1
vote
1answer
32 views

multicollinearity and interaction

I am a bit confused between multicollinearity and interaction. Would anyone mind explaining it to me and the difference between the two and how they affect the variables? I understand that ...
1
vote
1answer
32 views

Evidence that Collinearity Causes Predictors to be Insignificant

Question: Is there any evidence that collinearity causes some predictors in this model to be insignificant? Using R, I calculated the correlations of the predictor variables of a linear model. ...
0
votes
0answers
33 views

Using model comparison to deal with collinearity in linear mixed models

I am working on a set given to me, which involves fitting logistic models with a couple of predictors, some of which are nearly perfectly correlated (.9), imagine, face features deviations of specific ...
0
votes
1answer
16 views

Anova for potential colinear data

I have 4 columns of data, 1. Time (so time at 0, 6, 12, 24), 2. Drug types (A vs B), 3. Gene types (X, Y, Z) and 4. Values Now I’m trying to find whether drug A differs than drug B taking into account ...
1
vote
1answer
28 views

small T and large N, problem of collinearity in the model

If I am taking the log of a variable $X$ and also a square term of the variable $X$ on the right-hand side. Say, the equation is $$\log Y_{it} = α + β_1\log X + β_2 \log X^2 + u_i$$ where $Y =$ Waste ...
3
votes
1answer
63 views

Multiple Linear Regression with more variables than samples

I'm currently learning chemometrics for my work and I have a simple question about Multiple Linear Regression (MLR). Just to explain the context: I am simply using UV-Vis-NIR spectra (2500 wavelengths)...
0
votes
0answers
30 views

Logistic Regression with dummy variables? [duplicate]

I am working on a problem where response variable is binary and my features are dummy variables. I observed when I include intercept to model all the dummy variables' p-values are equal to 1. When I ...
0
votes
0answers
18 views

Multicollinearity in multiple linear regression with only categorial variables

I have to do a multiple linear regression with a numeric dependent variable and three categorial variables (2x2x4) as independent variables. Do I have to check for multicollinearity and if so, why and ...
0
votes
0answers
6 views

which test to do first? [duplicate]

I have a dataset that has 4 Columns as x axis. I want to check the data for: Autocorrelation, Multicolinearity, Heteroscedasticity and Normality, In what order should ı perform the tests? And ıf my ...
1
vote
1answer
29 views

What are the implications of the significance of two variables decreasing when in a model together and increasing when adding an interaction term?

In my data set, I have two primary predictors A and B. I also have the outcome Y. I have created four multivariate models that include other covariates. Model 1 includes A and covariates, Model 2 ...
0
votes
0answers
20 views

Negative variance inflation factor?

I'm trying to understand why I am getting so many negative VIFs when there shouldn't be any present to begin with. What would be the best way to diagnose the cause of this problem? I'm currently using ...
0
votes
0answers
18 views

Modeling the association between multiple correlated variables

I have a set of 10 variables all based around inventory optimization. Many of the variables are highly correlated. The ask is to determine the magnitude of the increase in one variable based on the ...
0
votes
0answers
19 views

macroeconomic regressors as xreg in ARIMA - differencing required?

I'm forecasting a timeseries that has both trend and seasonality component, which is why I am using ARIMA. Without providing external regressors, the best model selected (in training) has the ...
0
votes
1answer
115 views

Why do the coefficients of cross-sectional fixed effects and time fixed effects become zero?

I currently have a panel data set that contains the quarterly increment of loans initiated from more than 300 cities in China over the period from 2011Q1 to 2020Q2. I want to examine the impact of ...
0
votes
1answer
33 views

How many observations should a dummy variable encompass?

So I feel like this is a rather stupid question, but I can't find a straightforward explanation on the issue. When constructing a dichotomous dummy variable, how many observations should each category ...
4
votes
1answer
134 views

Is it okay to residualize a variable out of my dependent variable, to deal with multicollinearity?

Here is my situation. I have n predictors of interest, and two control variables. If I put them all together in a multiple regression, I get issues with colinearity (i.e., VIFs are very high, and the ...
2
votes
0answers
33 views

Can you use VIFs in bayesian models?

I have created a mixed effect bayesian GLM using rstanarm. I have a few parameters that I suspect to have correlation (or possibly collinearity) issues from looking at a simple correlation matrix. I ...
7
votes
1answer
278 views

Omitted Variable Bias (OVB) and multicollinearity

In a linear regression model, the reason we control for variables is to prevent the omitted variable bias (OVB). That is, suppose we are trying to fit the model $$ Y = \beta_{0} + \beta_{1}X_{1} + \...
3
votes
1answer
72 views

Getting the wrong sign on a coefficient in logistic regression?

I'm trying to make a logistic regression model explaining whether a law passed last year has affected my dependent variable. My most important variable (an indicator variable for whether the law was ...
1
vote
1answer
25 views

Testing for multi-collinearity after fitting a model with LassoCV from Sklearn in Python?

Is there a way to test for multi-collinearity, like VIF for example, after fitting a model with LassoCV from Sklearn in Python? https://scikit-learn.org/stable/modules/generated/sklearn.linear_model....
1
vote
0answers
22 views

Using a binary variable (present/absent) and its continuous counterpart (amount)

I am working on a banking-related logistic regression problem. Several of my variables are whether someone has an account of some sort, and there is a subsequent variable with the balance of the ...
1
vote
1answer
80 views

Convergence Error Using lifelines.CoxTimeVaryingFitter Python

I want to evaluate my Cox model using the lifelines package for a time varying covariate problem. However, when I use the lifelines.CoxTimeVaryingFitter I get a ...
0
votes
0answers
17 views

Does unbalanced data impact the Variance Inflation Factor?

I am interested in testing whether certain variables with a high correlation between them should be removed from the model. I was thinking of checking this out with VIF. I am working with a data set ...
0
votes
1answer
64 views

Working with systems with Perfect Multicollinearity

I am working with a time-series dataset that is based on demand-supply dynamics with several variables. THe sample data for one time period is: ...
0
votes
0answers
21 views

Working with systems with Perfect Multicollinearity [duplicate]

I am working with a time-series dataset that is based on demand-supply dynamics with variables: Production, Ending Stocks, Exports, Imports etc. I was working on a regression with this data and wanted ...
3
votes
1answer
31 views

Collinearity or not?

In a multiple regression analysis, if you have, in addition to host of other varibales, a dummy i.e. (0,1) coded gender variable with 0=male 1= female, and another dummy variable, say an occupational ...
0
votes
0answers
9 views

Addressing multi-collinearity issues with subset selection methods [duplicate]

I'm a long-time lurker and first-time poster to this forum... I am currently working my way through an Introduction to Statistical Learning, and I have a question regarding the algorithms presented ...
0
votes
0answers
9 views

Variable significance very sensitive to specification of non-correlated second variable

I´m doing research on a political science topic and my models leave me behind with a big questionmark at this point. I have a dataset containing 79 observations on a number of variables and trying and ...
0
votes
0answers
20 views

How does principal component regression help with multicollinearity?

In How to Deal with Multicollinearity?, the top comment in Aaron's answer says "These independent variables are now uncorrelated. Very low eigenvalues also indicate high degrees of ...
0
votes
0answers
35 views

Data with multicollinearity and p >> n

This is a csv file, the file is titled "res_final". The first line contains the names of the variables: ...
0
votes
0answers
10 views

Variance inflation factor vs condition number [duplicate]

We know the Variance inflation factor and condition number both help to measure multicollinearity. Which one should we use when?
2
votes
0answers
150 views

About spurious & masked relationship and multicollinearity

I'm reading Statistical Rethinking by Richard Mclreath and I am bit confused about the subjects in chapter 5. The book itself is about bayesian analysis. This chapter specifically point out to ...
2
votes
1answer
131 views

checking collinearity in a glm

I'm new to checking the VIF value for a glm model so I just want to make sure i"m understanding this correctly. I have 4 predictors for my count model and the model looks like this: ...
0
votes
2answers
39 views

Relationship between variables in a proposed model using linear regression?

I am new to linear regression and I am currently working on a linear regression problem - I have 8 features and one output. The features I am using seem unrelated to each other and I found an article (...
0
votes
0answers
28 views

Best subset regression with instrumental variable

I am applying multiple regression with a data. There are 19 regressors in total and one of them is endogenous. For the endogenous variable I have identified an instrumental variable. When I apply ...
0
votes
0answers
15 views

What is the meaning of the regressor characteristic root?

As described by Greene's Econometric Analysis (7th Edition), the regressor matrix's condition number measures how singular the matrix is. Therefore, the condition number is a measure of ...
0
votes
1answer
83 views

Regression with highly negative correlated variable [duplicate]

I have a regression with 6 independent variables and one of them is highly negatively correlated with one of them. (-0.81 correlation) I was wondering if the regression is still valid or if it's ...
1
vote
0answers
41 views

Testing multi collinearity of ordinal variables in R

I want to test if there is multi collinearity in my dataset. This is made up of multiple ordinal independent variables. Am I correct that this should be done as shown below ...
0
votes
0answers
21 views

How does Ridge regression / Regularization help in selecting less or more important features? [duplicate]

Can someone please explain how regularization helps to shrink the " less important " features to zero ? As far as I know , Regularization only penalizes the weights of ALL the features to ...
0
votes
0answers
22 views

Multicollinearity in ECM model

In Error Correction Model I have proplem related to singular matrix. There are multicollinearity problem of such variables as exchange rate and interest rate. Because they are somehow stable over time,...
2
votes
1answer
68 views

What happens to the coefficients of Ridge and Lasso when you have perfect multicollinnearity?

So let's say we ran a Ridge or Lasso regression on $Y \sim X$, and get coefficient $\beta_X$. Now if we duplicate the $X$, and call it $Z$, and then run the same regression on: $Y \sim X + Z$. How ...
0
votes
0answers
37 views

Does multicollinearity produce wrong beta estimates?

This is the first time for me to ask a question here. I'm sorry that if I break any rule here. I have encountered a problem about the consequence of multicollinearity. During reading the explanation ...
1
vote
1answer
68 views

VIF scores for ordinal independent variables

I suspected there was a high degree of multicollinearity in the independent variables of my data. Each of these variables is ordinal. The original model is ...

1
2
3 4 5
20