The multicollinearity tag has no wiki summary.
1
vote
0answers
19 views
Linear relationship between explanatory variables in mutliple regression
I was reading the multiple regression chapter of Data Analysis and Graphics Using R: An Example-Based Approach and was a bit confused to find out that it recommends checking for linear relationships ...
1
vote
0answers
15 views
Addressing multicollinearity with key driver analysis
I am trying to determine the key drivers from a series of 30 Independent Variables (IVs) (attributes rated on 10 pt scale) on 3 Dependent Variables (DVs) (i.e. purchase intent). The 30 IVs are pretty ...
1
vote
0answers
26 views
Incorporating non-collinearity into RANSAC affine transform estimation
The MATLAB code at this link can be used to estimate the affine transform between two sets of 2D points. The code makes the implicit assumption that neither of the point sets contain collinear points. ...
1
vote
1answer
41 views
Complex correlation in my regression?
I have two variables, the price of hay and insurance usage. Stata gives their correlation coefficient as 0.1227. This tells me that they are not highly correlated. When I have each variable in a ...
0
votes
0answers
43 views
Testing for autocorrelation of the residuals
I'm trying to test for autocorrelation in the residuals of an AR(p) model in stata using the command varlmar.
The stata output is: "the exogenous variables may not be collinear with the dependent ...
1
vote
1answer
51 views
How to eliminate high multicollinearity with a continuous moderating variable, and a categorical independent variable
I am looking at whether Corporate Venture Capital-backed firms (1) perform better than Independent Venture capital - backed firms(0) in their POST-IPO performance. My assumption is that this ...
1
vote
2answers
48 views
One predictor is a linear combination of another
If I try and fit the linear regression
lm(y~V1+V3,data=x)
with data:
...
2
votes
1answer
157 views
Why do coefficients and significance levels change so much in my OLS? Do I need MIXED model instead?
Please consider the following OLS model:
As a rule of thumb, a VIF score of max. 5 is considered acceptable collinearity in most fields. So for the better part I believed this model to be quite ...
2
votes
1answer
82 views
What is the fastest method for determining collinearity degree?
Given a large amount of data, how would I determine the degree of collinearity between all the variables? Preferably without relying on calculating linear regression between a variable and every other ...
2
votes
1answer
80 views
How to solve collinearity problems in OLS regression?
Please see the model below (link to bigger image). The independent variables are properties of 2500 companies from 32 countries, trying to explain companies' CSR (corporate social responsibility) ...
1
vote
1answer
37 views
Regression Estimation difficulties
My regression problem is properly formulated, but is encountering serious computational difficulties.
Dependent: $Y$ = multinomial
Independent: $X_1, \dots, X_{90}$ = linearly independent set of ...
0
votes
0answers
27 views
complicated model without a huge dataset
I've got a set of data that I'm trying to model. Lots of the data is missing, so I'm using multiple imputation. I've got about 360 observations and 13 variables. I'm also using GAMs, but that ...
0
votes
0answers
76 views
Multicollinearity: is a matrix scatterplot enough to rule it out?
I am analyzing an environmental data set containing one response variable (R) and many explanatory variables.
The explanatory variables are either factors (F1, F2, F3) or continuous variables (V1, ...
0
votes
0answers
102 views
Multicollinearity question
I have got a dataset with N=51. The dataset comprises two groups (n=26, and n=25).
I have one outcome variable and one covariate (both continues variables). I am interested in if there is an effect of ...
2
votes
0answers
97 views
VIF values and interactions in multiple regression
I am running a multiple regression of the form y~a+b+c+ab+ac+bc
I have checked the VIF values for the direct effects - should I check them for the interactions?
I am assuming not as that would ...
6
votes
5answers
306 views
What to do with collinear variables
Disclaimer: This is for a homework project.
I'm trying to come up with the best model for diamond prices, depending on several variables and I seem to have a pretty good model so far. However I have ...
1
vote
1answer
60 views
Non-independence of IVs in a random forest model
How is a random forest model affected if some of the variables are not independent?
1
vote
1answer
112 views
colinearity between variables
I am running a multiple regression of Y~a+b+c+d etc...
I want to do a quick check to see whether my different explanatory variables are colinear (they're a mix of categorical and continuous). ...
3
votes
2answers
260 views
How to deal with an unavoidable correlation between two independent variables?
In a simple linear regression model with two independent variables, if there is a strong correlation found between the variables, it is suggested that we should include only one of them in the model.
...
4
votes
1answer
108 views
Is there a reason to prefer a specific measure of multicollinearity?
When working with many input variables, we are often concerned about multicollinearity. There are a number of measures of multicollinearity that are used to detect, think about, and / or communicate ...
3
votes
1answer
240 views
Multicollinearity between categorical and continuous variable
Simplified version of my model:
glm(cbind(young, adults) ~ as.factor(month) + effort, family = "binomial")
i.e., I study proportion of young as a dependent ...
2
votes
1answer
180 views
Confusion related to multicollinearity, FA and regression of heterogeneous data
I am currently working with a data set that contains about 26 IVs of almost all sorts of scale of measurement (binary, nominal, ordinal and interval scale variables). There are strong reasons to ...
2
votes
0answers
72 views
Non negative least squares with minimal colinearity
I am trying to fit a dataset using the standard NNLS (non-negative least squares) approach.
Formally:
$\min_x ||Ax-b||^2_2$ s.t. $x\ge0$
This is a quadratic program and can be solved optimally. The ...
1
vote
0answers
75 views
Is high multicollinearity always an issue in OLS?
$$Y_t = a + bX_{1,t} + cX_{2,t} + dX_{3,t} + e_t$$
A high $R^2$ in $X_{1,t} = \alpha + \beta X_{2,t} + \gamma X_{3,t} + \varepsilon_t$ will always result in a higher standard error of the $b$ ...
3
votes
1answer
159 views
Multicollinearity in OLS
I am reading Greene's textbook Econometric Analysis where he says that, if there's multicollinearity, then:
Small changes in data lead to large swings in parameter estimates.
Coefficients have high ...
0
votes
1answer
114 views
Dealing with multicollinearity by separating common variance
On this page, I read about multiple ways to deal with multicollinearity. I determined that the last method suggested may be best for my purposes. I copied it below:
Treat the common variance as a ...
0
votes
0answers
58 views
Determining how much variance of Y is explained by X1 and X2 when you have multicollinearity?
In regression, when you have multicollinearity, how is it determined how much variance in Y is explained by the multicollinear X1 and X2?
2
votes
0answers
231 views
Dealing with multicollinearity when removing a highly collinear predictor reduces significance
In my experience with dealing with multicollinearity, often removing one collinear variable from the model results in the other collinear variable(s) becoming significant (assuming that all the ...
1
vote
0answers
49 views
Should I check Multicollinearity When There is An Inverse?
At Machine Learning algorithms there are usually inversion process about matrices and sometimes Matlab throws error when Multicollinearity occurs.
Should I check Multicollinearity(and how) everytime ...
0
votes
1answer
157 views
Regsubsets with leaps fails
I am trying to fit a problem with regsubsets with leaps in R. My problem is particularly strongly collinear, which is why I chose to use it in the first place.
...
0
votes
0answers
59 views
Why does NNLS only pick weakly correlated variables?
I have been looking to find an answer to this problem but haven't had much luck.
My problem has high multicollinearity and hence I am using NNLS (non negative least squares) in R. I am regressing ...
3
votes
2answers
120 views
Sign flipping when adding one more variable in regresion and with much greater magnitude
Basic setup:
regression model: y=constant+$\beta_1x_1+\beta_2x_2+\beta_3x_3+\beta_4x_4+\alpha C+\epsilon$
where C is the vector of control variables.
I'm interested in $\beta$ and expect $\beta_1$ ...
0
votes
0answers
137 views
Highly Collinear Independent Variables of Interest
Suppose I am interested in the follows:
I have county-level data. For each county, I know the share of the population that was born from one parent and the share of the population that was born from ...
0
votes
3answers
320 views
“Wrong Sign” On Regression Coefficients - Hierarchical Multiple Linear Regression
I am analyzing my data on the relationship between spirituality and negative emotional states (depression, anxiety, and stress) using a hierarchical multiple linear regression. Everything seemed to be ...
0
votes
0answers
62 views
Information criterion with penalty to condition number of hessian matrix?
I am working on a multinomial logistic model. I have the following questions:
AIC is asymptotically same as cross-validation, right? is it also true for logistic model? (N ~= 6000, p ~= 20)
I ...
2
votes
0answers
211 views
How to diagnose multicollinearity using the output of vif function in R?
I am running a logistic regression in R and am attempting to determine if multicollinearity is a problem with my model.
When I run vif() on my final model, I get ...
-1
votes
1answer
193 views
Determining eigenvalues and eigenvectors in R
x<-model.matrix(d) [,-1]
e<-eigen(t(x) %*% x)
e
values
[1] 1174600.548 21261.741 16133.842 6206.181 1856.894
First my $\alpha_1=1174600.548$, but ...
1
vote
0answers
31 views
Correlated factors in a global model
I have developed three a-priori candidate models that represent factors at three separate spatial scales. A factor in one model is correlated with a factor in another model. (there are debates in the ...
5
votes
1answer
1k views
How to deal with high correlation among predictors in multiple regression?
I found a reference in an article that goes like:
According to Tabachnick & Fidell (1996) the independent variables with
a bivariate correlation more than .70 should not be included in
...
2
votes
1answer
270 views
Two negative beta's in a curvilinear regression when mean centered or using standardized values
The problem I encounter is the following:
Imagine a (perfect) inverted U-shaped relation between an independent variable and a dependent variable. When you look at the curve estimation there is ...
3
votes
2answers
106 views
Overall prediction using correlated variables
I have a large set of data and a couple of regressors that seem to be somewhat to highly correlated. I will include these in a GLM and am primarily interested in the predictive ability of the model ...
3
votes
0answers
437 views
Time series forecasting using autoregressive and linear terms in R
I have real daily market data which I'm looking at to create a model for forecasting. The model that I created (below) used autoregressive terms within a linear regression.
I was sharing this with a ...
5
votes
2answers
500 views
Is multicollinearity implicit in categorical variables?
I noticed while tinkering with a multivariate regression model there was a small but noticeable multicollinearity effect, as measured by variance inflation factors, within the categories of a ...
3
votes
2answers
575 views
How to test for and remedy multicollinearity in optimal scaling/ordinal regression with categorical IVs
I have a data set containing only categorical variables (both nominal and ordinal in nature). The dependent variable is also ordinal (with 4 categories). I was planning to run a categorical regression ...
1
vote
1answer
94 views
Regression with averages and collinearity
I am stuck with the definition of a regression model regarding the inclusion of a constant averaged by the dependent variable.
I have a dataset M=(id, x1,x2,x3,x4) ...
5
votes
1answer
98 views
In a neural network with N covariates, are >N hidden units useless?
I'm fitting a neural network to this example data I found online:
Machine Learning Repository
I am cross validating 1 to 10 hidden units (in only 1 layer), and I have the minimum error with 10 hidden ...
0
votes
0answers
1k views
How to interpret R-squared in multiple regression with more sets of dummies and continuous variables?
I have a problem with a multiple regression I performed:
model without constant term;
one dependent continuous variable;
first set of dummies: derived from 2 continuous variables, I used the median ...
3
votes
1answer
221 views
At what VIF level should you switch from OLS to ridge-regression?
Regarding multicollinearity, is it recommended to use ridge-regression if you have some covariates with VIF values around 10 in the OLS model? What would be the best VIF level to use to decide ...
2
votes
1answer
125 views
Analyzing age-dependent variable with age-at-death as a predictor. Collinearity problems
Let's say I'm analyzing the incidence of pathological lesions, observed during necropsy. No information is available about when the lesion occurred before death. So, I'm using a general linear model ...
2
votes
1answer
352 views
PCA vs ridge regression for multicollinearity?
Orthogonalization via PCA and ridge regression are two common methods to account for multicollinearity for linear regression models. When would you use one over the other?
