Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
0
votes
0answers
59 views

Multicollinearity when estimating a gravity model

everyone! I am estimating a gravity model in order to analyze the impacts of tighter environmental regulations on international trade. More specifically, I am analyzing Brazil's trade flow. My (...
3
votes
2answers
65 views

Perfect multicollinearity with a cubic term in the model?

I'm trying to figure out why adding a cubic term in the model doesn't guarantee a perfect multicollinearity. If $X$ is known, then $X^3$ is known in both magnitude and sign and vice versa. It may not ...
1
vote
1answer
2k views

VIF for Categorical Variable with More Than 2 Categories

I'm trying to detect multicollinearity using VIF in both Python and R. Based on my knowledge, the VIF should be less than 10 if there is no multicollinearity. However, for the categorical variable ...
1
vote
1answer
53 views

How can I assess the strength of the collinearity (if any) between two different sets of categorical dummy variables in an OLS regression?

I ran an OLS regression of 'prices' on two sets of dummy variables and nothing else. The dummy variables in question are 'city', with levels A, B, C and D (each corresponds to a different city). And '...
1
vote
0answers
30 views

Using a group-level mean as a predictor in a mixed effects model

I want to fit a multi-level model across $j=1,2,3,...,k$ groups that predicts $y_{ji}$ for the ith observation in the jth cluster. $ Y_{ji} = \alpha_j + \beta_1 X_{ji} + \beta_2 \bar{X_{j.}} + \beta_{...
1
vote
0answers
124 views

Multicollinearity in Gaussian process regression?

Can data multicollinearity be a problem in Gaussian process regression? Multicollinearity in linear regression essentially makes it hard to determine which features are important in making ...
11
votes
1answer
337 views

Standardization of variables and collinearity

Collinearity can pose certain problems in various kinds of regression problem. In particular, it can make the parameter estimates have high variance and be unstable. Various methods have been ...
1
vote
1answer
82 views

Multicollinearity with only one variable per example?

I have a dataset were I know that there are nominal independent variables (IV) with multicollinearity (by theorethical knowledge about the IV). These IV have been created from a categorical variable ...
0
votes
0answers
29 views

Why the new variables formed by Almon distributed lag model are still highly correlated?

I just got started with the Almon distributed lag model. This is good reference I found that was very helpful and I basically followed the same methodology to create the "transformed" new variables. ...
1
vote
0answers
23 views

High, opposite estimators in presence of severe multicollinearity - what is the name of the problem?

Sometimes, when estimating models with two variables affected by severe multicollinearity they both become highly significant with huge estimators of opposite signs. For example +1001 and -1000. The ...
0
votes
2answers
1k views

Are feature selection methods prone to multicollinearity? [duplicate]

I'm using R and I wonder whether I should first remove highly correlated variables via vif()and apply then lasso, forward selection etc. Or doesn't it matter? Do ...
1
vote
1answer
101 views

Handling linear dependence among (non-mutually exclusive) binary predictors in linear regression

I have a number of movies tagged with varying genres. ...
1
vote
0answers
40 views

Do I have collinearity?

this is my first post on this forum. I'm a masters student and have a question regarding my experimental setup. I collected arthropods in six vineyards, each vineyard with two plots (a treatment and a ...
0
votes
0answers
44 views

Including multiple categorical predictors that are not independent in linear regression

I'm working with a model where we are studying a marker for a disease as well as risk factors, demographic variables, and related conditions. Marker: Marker concentration (continuous) Age: continuous ...
1
vote
1answer
31 views

Does it make sense to include GDP/capita and Population in the same OLS model

I got good results in my OLS model by using as explanatory variables: GDP/capita = GDP/Population and the Population. Do you think that is an issue to include both in the model (multicollinearity)?
0
votes
0answers
189 views

Ordinal multinomial logistic regression on one-hot encoded data

I have a task I am unable to tackle by principle. I'm working on survey data for one of our clients such that my design matrix is made of one-hot vectors with 15 features (originally 3 variables with ...
1
vote
2answers
90 views

Dealing with rank deficiency when multiple regressors are inherently related / a non-binary ratio

I am running a linear mixed effects model in which three of the regressor are inherently related. For sake of conceptual example: let's say I would like to see how the relative time employees arrive ...
0
votes
0answers
21 views

Do significant coefficients in a multiple regression analysis with moderate multicollinearity mean something?

I have a multiple regression model with moderate multicollinearity (VIFs <= 3.2). One of the coefficients is significant at the 10% level. Does this mean that despite of the variance inflation, I ...
5
votes
1answer
2k views

Mean centering interaction terms

I want to mean center my interaction terms in a regression model (i.e., make the mean zero for each variable). I understand that I am supposed to mean center my variables first and then multiply them ...
2
votes
1answer
66 views

Picking a model: Diagnostics or Model Strengths

I am building a lot of models and want to pick one to use for predicting. I am using linear regression, elastic net, and partial least squares regression. I know my data is highly correlated and that ...
0
votes
1answer
106 views

Regression shows high multicollinearity even after a PCA

I am comparing men and women using a measure of personality which has 24 variables. I did a oblimin rotated PCA for women and men separately so that they are truly representative of the population and ...
2
votes
0answers
176 views

What to do when predictors are proportions that sum up to one?

I have a situation as the one described in the links below: Interpreting proportions that sum to one as independent variables in linear regression Predictor variables sum up to 1 but not necessarily ...
0
votes
1answer
2k views

If the VIF is 2 then what is the value of correlation coefficient $R^2$

If variance inflation factor is 2 what is the value of correlation coefficient $R^2$? $$VIF = \frac{1}{1-R^2}$$ Given $VIF =2$, then is this calculation correct? $$\begin{align} 2 &= \frac{1}{1-...
2
votes
1answer
55 views

Controlling for variables in social sciences

I know this is a completely hypothetical scenario but I just want to understand how the effect of a variable could be held constant and how the coefficients of two independent variables are estimated ...
1
vote
1answer
98 views

Multicollinearity in simple linear regression

If there's perfect or near multicollinearity problem in a simple linear regression $y_i = a + b x_i + u_i$, what characteristics does $x_i$ have? I think if there's perfect multicollinearity, it ...
0
votes
1answer
49 views

Keeping two correlated variables in the model

I am using OLS: In my model I have two variable (X1 & X2) which are correlated (correlation = 0.47). My prediction is that X1 should be negatively associated with Y and X2 should be positively ...
0
votes
0answers
29 views

Collinearity among OLS regressors may inflate their variance, but can it also change their estimated value?

I've read that collinearity between independent variables in an OLS regression may inflate the variance of the OLS estimates. But, can it also change their estimated value?
0
votes
0answers
74 views

Collinearity when regressing against three sets of dummies

I would like to regress price of food products against three sets of dummy variables: 1. the food product itself (13 products) 2. the country where the food product was priced (119 countries) 3. the ...
0
votes
0answers
21 views

How to improve on the preferred regression model?

After running the chow test, the F stat shows structural change is present in the model, so the unrestricted models are preferred. I am not sure how I can choose from the two regression models in the ...
0
votes
1answer
161 views

Interpreting dummy variable interaction terms

I am attempting to model monthly retail electricity sales. To account for both the effects of seasonality and weather, I created an interaction term by multiplying 12 monthly dummy variables by the ...
0
votes
0answers
122 views

Interactions terms and the dummy variables

I am attempting to model monthly retail electricity sales. To account for both the effects of seasonality and weather, I created an interaction term by multiplying 12 monthly dummy variables by the ...
0
votes
1answer
60 views

Is “$\mathrm E[X'X]$ has rank $k$” the assumption of no multicollinearity?

My lecturer wrote this on the board: Assume $\mathrm E[X'X]=Q$ has rank $k$, where $X$ is the data matrix and $k$ is the number of independent variables. I asked her if that is the assumption of ...
1
vote
1answer
442 views

How is GVIF calculated for categorical variables?Also is there any other way to detect multi co-linearity of categorical variables?

I was tring to find a way to remove the redundant categorical variables as features. I believe GVIF would give high value for the redundant/multicollinear categorical variables. Please let me know if ...
1
vote
1answer
58 views

Interpretation of removed continuous variables in regression due to linear dependence

I have created a standard OLS regression model to estimate the House Price and one group of variables describe the age group percentage of population in a particular neighborhood (ranging 0 to 100). ...
1
vote
0answers
40 views

How do I classify multiple time series into different buckets?

I have 60 different time series, denoted $\{y_{i,t}\}_{i=1}^{60}$. So $i$ denotes the time series in question, and $t$ obviously denotes the time period from $t=1$ to $t=T$, where $t \in \mathbb{N}_{+}...
0
votes
0answers
19 views

R - checking colinearity of 3 categorial and 1 continuous variables

I have the following variables which are expected to influence the dependent variable kg waste: turnover (continuous), restaurant type (either D or I), operation (either P or N), owner (either M or F)...
1
vote
2answers
732 views

Choosing model for more predictors than observations

I'm working with a data consisting of 1000 observations of 2000 predictors and one variable we wish to predict. There are couple of problems I can't get around. I am aware that such setting has been ...
0
votes
1answer
1k views

How to overcome Coefficients: (4 not defined because of singularities) [duplicate]

Stats is not my strong point but trying to run a regression. I'm aware that it happens because some of these variables are perfectly collinear. However, I do not know how to fix this? Any help would ...
1
vote
0answers
522 views

Do control variables in a regression analysis cause collinearity?

This is something that bothers me for quite some time, but I didn't find yet a satisfactory answer. I hope that the wisdom of the people hear will help me to clarify this: In a multivariate ...
0
votes
0answers
339 views

Application of GAM on large dataset

I was suggested that my questions were too broad. As I commented below, I have nearly a million data points and perhaps a hundred variables. This may be a very basic modeling question: I am curious to ...
1
vote
0answers
177 views

Can I include an 'industry dummy' in my Relative Weight analysis? Bootstrapping produces error message!

I am analyzing the impact of 7 different employee satisfaction variables(x1, x2,...,x7) on financial performance in 200 companies over 3 years. Since these 7 predictor variables are highly correlated ...
-1
votes
2answers
72 views

Linear regression and multicollinearity

There is a multiple linear regression model being created. Y=ax1+bx2+cx3 Following HYPOTHESES are formed Variable x does not impact y for all variables x1, x2 , x3 and so on. We removed a ...
1
vote
1answer
264 views

Which multivariate test on repeated k-fold cross-validation with collinearity?

I am running repeated k-fold cross validation (5x5-fold) for comparing two models based on 3 dependent numerical variables (X, Y, Z) and 4 independent categorical variables (A (two groups), B (five ...
0
votes
1answer
418 views

Independent variable collinear with intercept

I have high collinearity between one of the covariates (credit score variable with values ranging from 600 to 800) and the intercept term, when regressing a continuous dependent variable on some ...
0
votes
0answers
24 views

Multicollinearity between categorical and continuous predictors [duplicate]

How can I check collinearity between categorical and continuous predictors (there are four predictors)?
0
votes
1answer
23 views

Year of intervention seems a better predictor than type of intervention (which is dependent on the year itself)

We have a small(n = 19) non-randomized pilot clinical study in which we compare two types of surgical procedures on various outcomes. The choice of which procedure was to be performed depended ...
0
votes
1answer
41 views

solutions for muticolinearity

how do you linearly combine predictors, such as adding them together to avoid multicollinearity? I have multicolinearity in my regression. say the function is lm(y ~ x + z) where x and z have ...
1
vote
1answer
100 views

Is repeated propensity score matching over many 0-1-features a valid procedure?

I would like to do a simple linear model where the outcome $y$ is real-valued, but my data matrix $X$ consists of several hundred features that all are $0$-$1$-valued. The number of observations $n$ ...
2
votes
2answers
60 views

Estimating effect of linear regression coefficients with multicollinearity

As I didn't find a satisfying for that questions I try it here: I have a multivariate Lineare Regression model with some correlated predictor variables. The "simple" question I want to answer is: "If ...
1
vote
0answers
42 views

Can a complex interaction term mean more than what it's composed of?

I'm cross-posting this question on both Economics and Cross Validated to get answers from a different perspective on each field. It is generally accepted to cross-post if the question is tailored to ...

1
3 4
5
6 7
20