0
$\begingroup$

I may be wrong but from my understanding logistic regression requires there to be little or no multicollinearity among the independent variables, and yet Kappa statistics as part of postResample() function in caret library (r) is a measure of reliability of the model. If we all agree that in Kappa 0 represents the amount of agreement that can be expected from random chance and that with 0 our model accuracy test is inconclusive, how can we start from the assumption of little or no multicollinearity among independent variables? Have I misread the entire model and failed to understand its basic pillars? Any useful answer is highly appreciated. Thanks

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Kappa is a measure of inter-rater agreement. Kappa is 0 when

Rating 1: 1, 2, 3, 2, 1

Rating 2: 0, 1, 2, 1, 0

because the two do not agree at all. But the two ratings have a correlation of 1.0, because they perfectly covary.

Multicollinearity is about the correlation (or covariance) among X's. Kappa is irrelevant here.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks bellmaneqn. Valid point. The main issue I have is about the reliability of the logistic regression model I built out of 3 variables using multinom() function from nnet package and delivering a 93% accuracy with Kappa 0 after calling postResample to validate its results. I am pretty sure there is no relationship explaining any variance with my predictors over the dependent variable which is not aligned with that 93% accuracy result delivered. Thanks $\endgroup$
    – AlGrasso
    Commented Oct 15, 2018 at 13:44
  • $\begingroup$ I am not sure if I understand your situation. So you have a model with a Y and two X's. Now you built a logit of the two X's to predict Y and you get 93% accuracy. What is the Kappa = 0 about? Kappa of the two X's? $\endgroup$
    – BellmanEqn
    Commented Oct 15, 2018 at 13:58
  • $\begingroup$ In this instance Kappa is one of the output of postResample function and presumably is calculated between the values (0 or 1) that can be assumed by the response categorical/binomial variable Y. There is an overwhelming majority of 0 in the observations of the entire dataset (which could account for high accuracy) but the independent variables have close to 0 incidence to Y which made me raise the question in the first place $\endgroup$
    – AlGrasso
    Commented Oct 15, 2018 at 15:06
0
$\begingroup$

To put it more broadly, multicollinearity can be an issue when it comes to either linear or generalized linear models (the latter including logistic regressions).

First of all, are you using the Kappa statistic as a measure of multicollinearity? If so, then you would be better off using VIF (variance inflation factor), which is included in the car package.

The Kappa statistic measures inter-rater agreement for qualitative items, but you can run into issues if some of your independent variables are in interval format.

Suppose you have the following regression equation:

reg1 <- glm(y ~ x1 + x2 + x3, data = mydata)

To test for VIF, you can simply apply:

vif(reg1)

In general, if you obtain a VIF statistic below 5 for a variable, then you can make the assumption that this variable does not suffer from multicollinearity and does not need to be discarded from the model. You should observe the theoretical meaning behind each independent variable and how it relates to the dependent, as there can be cases where it makes mroe sense to retain a multicollinear variable if it is of importance in explaining variation in the dependent variable.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi Michael, thanks for your prompt and insightful reply. In my dataset I have 3 variables: 2 predictive X made of quantitative continuous variables and 1 categorical binary dependent variable Y (following the dictates of the logistic regression). After calling postResample() I got an accuracy of the model of 93.18% (which sounds too good to be true) with Kappa value at 0. I am confident of the low incidence of the predictors (X) over Y (as also confirmed from the type="class" argument in the predict function) which made me wonder: how can I have such high accuracy from such variables? Thanks $\endgroup$
    – AlGrasso
    Commented Oct 15, 2018 at 13:35
  • $\begingroup$ Are you using the Kappa value to detect multicollinearity in its own right? Also, when you are detecting accuracy have you split the data into training and test partitions? If not, you might be overfitting. I would recommend using VIF to see if multicollinearity is present in the first instance, and you'll be better able to work from there. $\endgroup$
    – Michael Grogan
    Commented Oct 15, 2018 at 16:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.