Questions tagged [multicollinearity]

Situation when there is strong linear relationship among predictor variables, so that their correlation matrix becomes (almost) singular. This "ill condition" makes it hard to determine the unique role each of the predictors is playing: estimation problems arise and standard errors are increased. Bivariately very high correlated predictors are one example of multicollinearity.

Filter by
Sorted by
Tagged with
0
votes
2answers
24 views

Multilinear regression with nominal predictors - Bayesian

I have four nominal predictors and one metric predicted variable. I would like to know which one of predictors have more influence on the predicted variable. For doing so, I am curious to know if I ...
0
votes
0answers
6 views

How to fit the data with both Ridge and Robust regressions

I have a set of data that has multicollinearity and heteroscedasticity. I understand if the data only have multicollinearity we can use ridge regression or I can use the VIF indicator and remove the ...
0
votes
1answer
22 views

Multicollinearity for Offset Variable?

In a Poisson model, do we need to check for multicollinearity between the offset variable and other covariates? For example, if I offset by population size, would it be problematic to include ...
2
votes
1answer
38 views

How does multicollinearity affect the eigenvalues of a matrix?

I have been looking into ridge regression as a method to address multicollinearity in data. I am aware that multicollinearity can cause high variance in coefficient estimates. I have seen equations ...
1
vote
0answers
15 views

Do we need to do multicollinearity check if we are building a lasso/ridge/Elastic net regression models?

I have built an elastic net model for classification purpose, but I haven't done multi-collinearity check. Would doing multicollinearity check and then feeding the variables to the model have an ...
5
votes
2answers
208 views

What is finite precision arithmetic and how does it affect SVD when computed by computers?

Was reading the paper "DETECTING AND ASSESSING THE PROBLEMS CAUSED BY MULTICOLLINEARITY:A USE OF THE SINGULAR-VALUE DECOMPOSITION" by David Belsley and Virginia Klema. After performing SVD, while ...
1
vote
2answers
33 views

Why do sources claim that linear regression assumes little to no multicollinearity, among other “assumptions”?

Many sources (here is one of many https://towardsdatascience.com/assumptions-of-linear-regression-algorithm-ed9ea32224e1) state that linear regression assumes there is little to no multicollinearity ...
0
votes
0answers
16 views

How can I combine a binary and discrete variable to avoid multicollinarity? [closed]

In my model, I have two variables which might cause collinarity problems. Variable A: binary variable Variable B: discrete variable To check for collinarity between these two I used several methods: ...
1
vote
3answers
44 views

How correlated do regressors need to be to violate the collinearity assumption?

One of the assumptions of standard OLS regression is that the regressors are not correlated. But what is the level of correlation at which the assumption is violated? So for instance, if I have three ...
0
votes
0answers
23 views

Removing multicollinearity from dataset VIF or regularisation

Hi I am quite new to regression analysis and would appreciate some guidance on multicollinear data. I have a dataset with 28 variables and 250 observations. What I want to do is use one of these ...
1
vote
0answers
16 views

Regularization and multicollinearity

Multicollinearity/collinearity, from what I understand, occurs when 2 or more predictor/independent variables (variates) are strongly linearly dependent. This leads to overfitting, and we could use ...
0
votes
0answers
27 views

Including interaction term without main term with possible aliasing

I'd like to model an interaction term between a continuous variable and categorical variable, while accounting for possible aliasing in the variables. I was wondering what the best way to do this was. ...
0
votes
0answers
5 views

Combined multilevel factor predictor vs interaction terms of categorical variables generalized mixed effects model

I am modeling the effects of three factor variables on various response variables using generalized mixed effects models with glmmTMB in R. Factor 1 has 4 levels, factor 2 has 3 levels, and factor 3 ...
0
votes
0answers
7 views

Calculate Variance Inflation Factors for WLS (using python package?)

Is there an easy extension of VIFs (variance inflation factors) to WLS regression, hopefully easily available in Python? I have an application where I am optimizing the operation a system, modeled as ...
1
vote
0answers
33 views

What is the co-linearity cutoff between variables that starts to make LASSO predictions unreliable?

I read several posts and resources including this, this, and this. I understand the colinearity problem in machine learning. I also get why LASSO method becomes unreliable if the predictor variables ...
0
votes
4answers
107 views

What is meant by multicollinearity does not influence the predictive power of the model? [duplicate]

1)I saw many posts online and on cross-validated itself, that the predictive power of a model is not influenced by multicollinearity. I would like to know what is meant by this statement, does it mean ...
1
vote
1answer
26 views

Ridge-like alternative for multicollinearity in classification

Multicollinearity affects both regression and classification tasks. OLS is the standard model for regression, while Logistic regression is the standard for classification. Ridge regression is a ...
0
votes
0answers
27 views

Independent Variables With Probabilistic Values

I am currently working with data where the independent variable is in the form of a probability (specifically outcomes of FCM clustering). In the data, there are 8 different clusters and each ...
0
votes
0answers
16 views

Why and how adding interaction terms to regression equation let me to detect possible collinearities in predictors?

First of all, I should say that I am a computer scientist but I don't have a solid background in statistics. In one of the works I've done, I was using multiple linear regression with continuous and ...
2
votes
0answers
12 views

About multicolinearity in Interaction Terms

Dears, In my panel data analysis, I am using interaction term. The model looks something like Y=b0+b1*X+b2*Z+ b3*XZ+e. Now the interaction term is strongly correlated with X or Z. In order to get ...
0
votes
0answers
13 views

Selecting variables using forward selection method

If we are to select predictors for a regression model using forward selection and the information available to us is just a correlation table, do we select the predictors that have a strong ...
1
vote
0answers
12 views

Correlations between categorical and continuous variables in a mixed effect model

I have a dataset with two independent predictors, one categorical (6 levels) and one continuous. I'm also including an interaction term and a random effect of site. $$ yij=β0+β1*x1ij+β2*x2ij+β3*x1ij*...
1
vote
0answers
55 views

Using AIC to compare models with collinear explanatory variables

I'm using logistic regression to test whether the proportion of seeds to germinate across an elevational gradient is explained by temperature, precipitation, elevation, or other site features that I ...
1
vote
0answers
34 views

Multivariate regression model that has two highly correlated variables, but performs “better”?

I have two variables (years_in_city and in_city_since) that have an r-squared of .97 between each other. years_in_city is calculated by taking the current year and subtracting in_city_since from it, ...
0
votes
0answers
16 views

Multicollinearity in structural equation modeling with multiple imputation?

Using R, I created a structural equation model and fit it to multiple datasets using the 'sem.mi()' function from the SemTools package. I know multicollinearity tends to be a concern for structural ...
0
votes
0answers
55 views

“aliased coefficients”

I tied to make make a "linearHypothesis" to test joint significance with the "car" package in R. However I got the error massage "there are aliased coefficients in the model." My regression runs over ...
1
vote
0answers
25 views

Why does Ridge Regression work better than LASSO in the presence of multicollinearity? [duplicate]

I am learning the Ridge Regression. And I found that Ridge Regression is better than LASSO to analyze multicollinear? I wonder the reason?
0
votes
0answers
10 views

Is multicollinearity of independent variables a problem in time series analysis

I am performing a time series analysis (co integration and var) to analyse the effect of government investment on real gdp. I could not found a direct variable for government investment because of the ...
0
votes
0answers
15 views

Is a regression model the best model to use for predicting stock prices based on other stocks considering the multicollinearity problem?

I'm just learning how to use Sklearn and python for data analysis. One of the projects I'm working on is trying to predict the price of a stock over a short time span (a few minutes) based on the ...
0
votes
1answer
17 views

Using prior knowledge about correlated variable in ridge regression

I am wondering what methods are available for incorporating prior knowledge of some variable that is correlated with the unknown regression coefficients in a ridge regression. I have a sparse matrix ...
0
votes
2answers
17 views

Removing Multi-Collinearity Reduces F1 score

I was trying to build a classification model and I found that the features were highly correlated. I tried run a random forest model on the features and got an F1 score of 0.44 but when I removed the ...
0
votes
0answers
12 views

Multicolinearity in logistic regression using final fit package in R

I ran a multiple logistic regression model (second screenshot) using final fit package in R. I read that if there is big difference in values of univariate and multivariate ratios then there might ...
4
votes
2answers
106 views

Omitted variable bias vs. Multicollinearity

There's seems to be a bit like catch 22: suppose I am doing linear regression, and I have 2 variables that are highly correlated. If I use both in my model, I will suffer from multicollinearity, but ...
3
votes
2answers
101 views

Physically implausible results in linear regression with colinearities

While developing a model (a Poisson regression, but this is not the topic of this post), I stumbled upon a physically implausible relationship between some variables. I have ground temperature data ...
0
votes
0answers
19 views

Do I go with theory or statistics when assessing partial collinearity?

I am performing multiple regression on a dataset which has two continuous predictors 'A' and 'B'. The correlation between predictor A and predictor B is -0.7 and the VIF is <5 i.e generally below ...
0
votes
1answer
29 views

Checking Multicollinearity and building a classification model when dependent is a factor and other independent variables are numerical in r

Problem statement Y - Dependent variable is a factor (with levels A, B, and C) Independent variables are all numerical variables. Important: I have only 70 data points. End Goal: Building a ...
0
votes
1answer
23 views

High Variance inflation factor for all variables

As you can see there vif for all variables is too high...My interpretation is that all variables are highly correlated to each other. How can I further investigate on this? Or how can this be further ...
0
votes
1answer
33 views

How to code variables to avoid singularity in quasibinomial glm model of pre/post test means?

I'm performing a meta-analysis of studies that compared treatment and control groups from pre-test to post-test using means of count data. Other authors have performed similar analyses using raw ...
0
votes
0answers
23 views

Cox regression with fully factorial experimental design

I am currently trying to evaluate data from a study we performed using a Cox regression (from survival analysis). First off, let me apologize for probably using wrong words in some places - I am not ...
0
votes
0answers
11 views

How to address multicolinearity caused by large number of zeros in the predictors?

this is the first time I am posting a question here, so please forgive me if it is out of place. Basically I am trying to predict user behavior in an online forum. My main predictor is their topic ...
0
votes
0answers
6 views

drop one of the terms involved in an interaction

I took an online course and the instructor dropped one of the terms in an interaction to remove multicollinearity. I'd previously been taught to always keep both terms involved in an interaction and ...
2
votes
1answer
84 views

How to handle strongly correlated but not perfectly collinear dummies

I am using several dummy variables in a GLM model implemented in R with a logit link function. However, the coefficient of one of the dummy variables is not shown in the results with the warning ...
1
vote
0answers
20 views

Which model fit best colinear indepent variables?

I have one dependent variable ($y$) and $200$ linear independent variables. However, some of them are related (exists multicollinearity). Therefore, I can't use multivariable linear regression. I ...
13
votes
2answers
858 views

Why doesn't collinearity affect the predictions?

I have read in many places that collinearity doesn't affect the predictions. It only affects the coefficient tests and confidence interval. As a result it cannot be used for causal inference but for ...
1
vote
1answer
20 views

Regression with both Original value and log transformation

In a regression, there is a independent variable x, say x is a positive number indicating number of months. Is it appropriate to include both the original value(X) and its Log transformation(logX) as ...
0
votes
0answers
16 views

Multicollinearity and OPLSDA fail

Which one would be the alternative if I find this problem? I am performing OPLDS-DA to determine, among my 58 parameters (104 observations), which one(s) drive the separation between my disease ...
0
votes
1answer
46 views

Use of collinearity in model building for regression

How should I use concept of multicollinearity to decide which variables to consider for building a model from a given data. how should i know what should be the correlation value to be considered ...
0
votes
0answers
10 views

How many groups of dummy coded variables can be included within one step of Hierarchical Regression?

I am conducting a hierarchical multiple regression analysis. I have several predictor variables which are nominal and ordinal. From what I understand, these need to be coded into dummy variables (each ...
0
votes
0answers
151 views

Mean-centering changes the values of regression coefficients with interaction terms [duplicate]

To my knowledge, mean-centering does affect the values of regression coefficient for variables involved with interaction terms. But it does reduce the standard errors of the coefficient estimates for ...

1
2 3 4 5
18