1,534 reputation
1037
bio website
location
age
visits member for 3 years, 6 months
seen 15 mins ago

8m
asked How do I find a p-value of smooth spline / loess regression?
1d
awarded  Popular Question
Jan
11
awarded  Nice Question
Jan
11
awarded  Notable Question
Jan
9
comment What does AUC stand for and what is it?
can you please explain the ROC curve in the context of a simple crossvalidation of the 0/1 outcome? I don't know understand very well how the curve is constructed in that case.
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
@kjetilbhalvorsen so are you suggesting that standard R functions like solve, ginv and is.singular.matrix (see EDIT 3) are all buggy? Not very likely, so please re-think your statement. You are welcome to look at the matrix I present in EDIT 2 and check it yourself!
Dec
30
revised Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
added 13 characters in body
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
@kjetilbhalvorsen the linear dependence is not approximate - the variables sum exactly to 1.
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
+1 thanks for the lasso tip! As for dropping the column, I don't think this is a good solution. If the habitat I drop is important for abundance of the species (response variable), I will loose its prediction power and significance of this coefficient (since all the other ones will have no important meaning for the species).
Dec
30
revised Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
added 109 characters in body
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
@gung, the definition of perfect multicollinearity at wiki says that the matrix is singular and cannot be inverted, which is not my case - see EDIT 3. How can I understand this?
Dec
30
revised Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
edited title
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
1) The definition is wrong because it messes up linear dependence and correlation - it implies that the variables must be correlated, which is not true, as I have shown. I would like to see a single consistent definition which we can use. 2) I don't know what you mean with "high level of correlation", there is none in my generated matrix in EDIT2. 3) The definition of "perfect multicollinearity" at wiki is that the matrix is singular and cannot be inverted, which doesn't apply to my case either. So how can I see that the definitions you are proposing apply to my case?
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
@Aksakal see EDIT 2.
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
Thanks. 1) But why do they speak of correlation? The definition is clearly wrong 2) it is interesting that the matrix can actually be inverted, please see my EDIT 3.
Dec
30
revised Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
added 320 characters in body
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
It is interesting and strange that the matrix can be actually inverted: if I do solve(t(as.matrix(envV)) %*% as.matrix(envV)) both on my real predictors and the artificial generated ones in EDIT 2, the inverse matrix will get computed with no error or warning (both with solve and ginv())
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
1) I am the OP :) 2) First sentence of the definition of Multicollinearity at wikipedia says "statistical phenomenon in which two or more predictor variables in a multiple regression model are highly correlated" - which they are not, see my EDIT 2
Dec
30
comment Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
I am afraid you are messing up linear dependence with correlation (collinearity). The fact that the variables sum up to 1 doesn't mean they are correlated - see the example I've added in the edit.
Dec
30
revised Predictor variables sum up to 1 but not necessarily correlated - is it a problem?
added 1137 characters in body