# Tagged Questions

Multicollinearity means predictor variables are correlated with each other, making it harder to determine the role each of the correlated variables is playing. Mathematically, it means the standard errors are increased. Multicollinearity can have counter-intuitive effects.

28 views

### Centering variables in regression leads to the same model of original variables, why still doing that?

The regression model y= b0+ b1 x + b2 x^2 + b3 x^3 and the second regression model y = b0 +b1 (x-u) + b2 (x-u)^2 + b3 (x-u)^3 where u is the mean of x These two models lead to the same curves, or ...
34 views

### What are some of the potential consequences of adding junk controls in your regression?

Let's say I am running a regression in which my dependent variable is homicide and my variable of interest is access to violent videogames. Let's say that I also throw in the kitchen sink with ...
14 views

### Cause of omitting independent variable in regression

As you can see in the output below DNTN is omitted but it's still in data What's the cause of omitting independent variable in the regression when the number of ...
18 views

### How to show the equality of two Variance Inflation Factor(VIF) definitions

How to show the diagonal elements of the ${C = (X'X)^{-1}}$ are $C_{jj} = \frac{1}{1 - R_{j}^2}$ where $R_{j}^2$ is the coefficient of multiple determinations from the regression of $x_j$ on the ...
39 views
+50

### Where is the explanatory effect of common variance among covariates accounted for in regression procedures?

As a follow up to the excellent answers provided for: Does the order of explanatory variables matter when calculating their regression coefficients? (Which I've found incredibly useful from a ...
21 views

### Should potentially multicollinear variables be dropped

I did correlation analysis for a set of variables and calculated the VIFs. For a couple of independent variables, the VIF is less than 4, however the correlation coefficient with other input variables ...
42 views

### When 2 variables are highly correlated can one be significant and the other not in a regression?

In regression, when 2 parameters are correlated and added to a model separately, how likely is it that one parameter will be a significant predictor of the response variable while the other is not? ...
14 views

### Should I use mean-centering on predictors for collinearity involving the intercept?

In one of my linear regression models, a predictor showed collinearity with the intercept based on condition indexes and variance decomposition proportions diagnosis. Then I found that this predictor ...
36 views

### Motivation to center continuous predictor in multiple regression for sake of multicollinearity?

I'd like to discuss the centering of continuous predictor variables in multiple linear regression with an interaction term for the sake of "relieving" multicollinearity. I've read about ...
51 views

### Why is multicollinearity a sample phenomenon?

We say that multicollinearity is a sample phenomenon. That means we postulate the PRF such that each independent variable is bound to have an independent effect on the dependent variable but due to ...
29 views

### Multicollinearity in WLS regression

I run a weighted-least squares regression to account for the heteroscedasticity in my data. When I examine the Pearson correlations between all predictor variables, I can't detect high collinearity. ...
31 views

### gam smoother vs parametric term (concurvity difference)

I have a gam model that is: gam=gam(sv~s(day,bs="tp")+s(range,bs="tp")+s(time,bs="cc"),data=train.all,gamma=1.4,method="REML") the ...
25 views

### Multicollinearity in Auto Regressive models

I have just started learning about time series analysis. I had a doubt regarding AR models. I understand that in Auto Regression, we regress one variable on values of the same variable at different ...
30 views

### Interpreting OLS Regression Coefficients with High Multicolinearity

I am having trouble understanding the interpretation of OLS coefficients when predictors are highly correlated. My understanding of OLS coefficients is that they estimate a change in the expected ...
48 views

### How to interpret the result of alias on a lm model in R?

I found that the lm model I have trained has some NAs in the coeficents, and it is in the factor columns. So I have searched for aliases an I have found that some factor values depend on the other ...
18 views

### Dealing with multicollinearity by regressing on residuals

I need to perform logistic multiple regression. There some highly dependent variables as income and expenditures, work experience and age, etc. Is it OK, for example, to run regression of work ...
17 views

### Large condition number, but no effect on validation

My question is in regards to the relation between the condition number (or other multicollinearity diagnostics) and validation of a linear regression (lm in R). Specifically, I have extremely high ...
45 views

### Why is a correlation coefficient threshold of r = 0.6 among predictors commonly used in ecology?

In my field of study (wildlife ecology), a correlation coefficient of r = 0.6 is a commonly-used threshold for identifying collinearity among pairs of predictor variables. In other words, predictors ...
46 views

### Mean and residual centering, orthogonal polynomials and effect coding

I understand that in order to be able to interpret main effects / lower order terms in a more intuitive way in linear regression models with interaction effects one should either use mean or residual ...
28 views

### Interpretation of simultaneous and independent ordinary least squares regression

I'm using ordinary least squares to regress a noisy overdetermined system. $$y = \beta_0 x_0 + \beta_1 x_1$$ For comparison, I'm also solving the independent equations \begin{align} y &= ...
52 views

### How to model partially nested & partially non-nested data in a mixed model?

How to model the data structure (correlation matrix / multicollinearity) for my dataset (described below)? Will it allow me to run a mixed model? At our website, we have pages with a block of ads ...
18 views

### Coding Social Influence Logistic Regression

I am fairly new to regression modeling and was hoping to get some help. Forgive me if this is has been asked before, but I couldn't find anything (maybe I was using the wrong keywords) I have a ...
486 views

### Set of uncorrelated but linearly dependent variables

Is it possible to have a set of $K$ variables that are uncorrelated but linearly dependent? i.e. $cor(x_i, x_j)=0$ and $\sum_{i=1}^K a_ix_i=0$ If yes can you write an example ? EDIT: From the ...
22 views

### How to deal with multicollinearity in a Translog Production Function

What are the different methods for dealing with multicollinearity in a translog production function? I have seen several methods such as: Checking for variance inflation factor (VIF) and ensuring ...
387 views

### What are the merits of different approaches to detecting collinearity?

I want to detect whether collinearity is a problem in my OLS regression. I understand that variance inflation factors and the condition index are two commonly used measures, but am finding it ...
73 views

### Collinearity in R for dataset with 40+ variables?

I have a big data matrix with 6000 rows (observations) and 45 columns (44 predictive variables (categorical and continuous) and 1 response variable (0 or 1). I want to check the correlation/ ...
14 views

### Is covariate adjustment appropriate in this case?

I have two conditions, A and B. In condition A, my covariate is always 0. In condition B, it is always > 0. The covariate is the number of "bonus coins", and it happens at random times (every 0-15 ...
70 views

### Difference Endogeneity and Multicollinearity in Logistic Regression

I am right now working with logistic regression and test my model over and over again. However, I am still not sure about the terminologies endogeneity and multicollinearity. For my under-standing, ...
42 views

### Multicollinearity in case of nonstationary variables

I am using dynamic OLS regression and fully modified OLS regression (in eViews) to estimate long run relationships' coefficients; my variables are all I(1). Do I need to check for multicollinearity ...
37 views

### Can I have an IV which does not have main effect on DV directly but might have an interaction effect with another IV on DV?

I am inducing envy (IV 1) to see the effect on focusing illusion/anchoring bias (DV). Since I am going to induce envy by showing attractive others' pictures,then gender will play a role because ...
67 views

### How to interpret a VIF of 4?

I am doing a multiple regression, trying to test the extent to which personal income changes and Presidential popularity can predict election results. I have a small sample size, unfortunately, as ...
44 views

### BIC difference for model selection when models have different (and correlated) predictors

I have a binomial dependent variable Y and two main IVs: A is categorical (5 non ordered levels) and B is continuous. A and B are collinear (I tested the effect of A on B with a linear model, and it ...
1k views

### Why is multicollinearity not checked in modern statistics/machine learning

In traditional statistics, while building a model, we check for multicollinearity using methods such as estimates of the variable inflation factor (VIF). But in machine learning, we instead use ...
59 views

### Variable correlation and collinearity in logistic regression

Hoping to get some insight on the issue of correlation/collinearity in predictors for logistic regression. Let me preface this by saying I’m no statistician, but rather a GIS analyst with exposure to ...
23 views

### How can I check for collinearity and heteroskedasticity in mixed models with lme (package nlme)?

How can I check for collinearity and heteroscedasticity in lme (R package nlme)? I found a blogpost that provides functions to do so for package lme4, but not for nlme. Here's a minimal example ...
82 views

I am doing a study for my masters correlating two separate development indicators to election results for the incumbent government. Unfortunately, I was only only able to get 11 years worth of data ...
64 views

### Regression with two dummy variables: city and state in which one is implicitly included in the other: multicollinearity?

I am new to statistics and am really struggling with something that is probably easy for a statistics expert to answer. I am currently working on my dissertation in which I am trying to see whether ...
13 views

### 4 continuous DVs, 12 continuous & discrete IVs: Issues of multicollineararity & multivariate question

I am conducting an analysis to see if various attributes about a set of stimuli affect the rates of different kinds of errors subjects make. I have 4 continuous DVs (error rates) I'd like to predict ...
151 views

### Multicollinearity problems with polr function in the MASS package for ordinal response [closed]

I've been trying to use the polr function for a couple days now. The dataset has lot of features (~70) and some of them are factor variables. When I run a simple ...
31 views

### Break an independent variable into two and collinearity? How to interpret the result

I am not sure if the title makes sense. Here is my situation. I am running a regression as below: $$y = \alpha_0 + \alpha_1 T_1 + \alpha_2 Z + \epsilon$$ Where $T_1$ is my interested covariate: ...
45 views

### Model selection in mixed-effects model with collinearity trouble

In a model aimed to assess the influence of land use measures on ecosystem functioning, I have one log-transformed dependent variable (the ecosystem function), and 5 fixed-effects independent ...
35 views

### How to deal with different outcomes between pairwise correlations and multiple regression

I have different results from a correlation table and a multiple regression model. I know that it is an effect of multicollinearity because correlations up to $.474$ exist between predictors, but this ...
38 views

### Detect multicollinearity in maximum likelihood scenarios

I'm estimating a binary logit discrete choice model with BIOGEME and want to check for multicollinearity of my predictors. BIOGEME uses maximum likelihood estimation (MLE) and not ordinary least ...
24 views

### Variable selection with longitudinal and correlated data

I'm working with a high-dimensional medical database, with detailed monthly medication reimbursement data as well as occurence of diverse medical outcomes, over several years (2010 to 2013). My ...
321 views

### Why is post treatment bias a bias and not just multicollinearity?

In this presentation by Gary King, he discusses post treatment bias as follows: Post treatment bias occurs: when controlling away for the consequences of treatment when causal ordering ...
58 views

### How can I check whether multicolinearity exist between categorical variables or numerical and categorical variables?

I did a linear regression with 10 variables, including categorical and numeric variables. But although my $R^2$ was 0.8 there were only 2 variables that were statistically significant. Am I correct ...
66 views

### If x2 & x3 affect x1, & x1 affects y, should x2 & x3 be included in a regression model?

Let's consider the regression $y=x_1+x_2+x_3+\varepsilon$ It is known that $x_2$ and $x_3$ affect $x_1$, but $x_2$ and $x_3$ do not affect $y$. $x_1$ can affect $y$, but only to a small extent. The ...
71 views

### Adding up regression coefficients in the presence of multicollinearity and reversal of signs?

My time series model suffers from multicollinearity between two independent variables. When taking one of the variables out of the model I obtain a coefficient of 0.08 for variable 1 but when ...