1
$\begingroup$

I have a dataset which consists of mainly categorical predictors (n=7) and also some continuous ones (n = 3) and a binary outcome variable. I have previously made a classification model using logistic regression. This model performs well with an AUC on a testing set of 67%. I have the option of breaking down the binary outcome into multiple outcome categories and I have reason the believe that this will increase prediction accuracy. This final model will also have to be implemented in excel (though I build my models in R). From reading Introduction to Statistical Analysis by Trevor Hastie, and from this article: https://help.xlstat.com/customer/en/portal/articles/2062225-discriminant-analysis-in-excel-tutorial?b_id=9283 it appears that Linear Discriminant Analysis is the clear choice. However, from this post: Can we use categorical independent variable in discriminant analysis? it appears that LDA doesn't handle categorical input variables. The best multi-class classification algorithm would probably by a neural network, however that would be too difficult/impossible to implement in excel. What approach can I take to build a multiclass classification model based on categorical and continuous predictors? Thanks

$\endgroup$
1
$\begingroup$

I think categorical variables can be handled in LDA as well. You can do label encoding or one hot encoding for the categorical variables and then you're good. You can also try SVM for multi class classification problem if your dataset is not that big. Ps: I won't say that AUC of 0.67 on testing set implies that model performed well. Also, Neural network is generally preferred when you want to find complex patterns and a large amount of data is available.

$\endgroup$
2
  • $\begingroup$ Thank you for your response. However, on what basis do you believe that a neural network performs best? From my experience the probability that a neural network outperforms LG is at chance. Also the value of an AUC depends on the nature of your variables. When relating socio-demographic variables to loan repayment 67% is actually higher than usual. $\endgroup$ Aug 18 '17 at 7:55
  • 1
    $\begingroup$ @stevezissou, I don't have much experience with neural networks and I have also seen it performing not so well on binary classification problem, but it might be helpful when you are dealing with multi class classification problems. Neural network is better in capturing the non-linearities present in the problem and that's something I read over the Internet. But again, it always depend upon the problem you are dealing with. $\endgroup$ Aug 18 '17 at 14:26
0
$\begingroup$

I would say against the accepted answer by Harshit [See the first point below]:

  1. Mathematically, categorical predictors are not allowed in discriminant (function) analysis as LDA assumes multivariate normality of predictors. Even in tools like SPSS, you can add categorical predictors for LDA but it doesn't mean it's a good practice to do that.
  2. There is no best model/algorithm. You need to try different models based on your data and see what works best. You can use cross-validation for model selection. All models are wrong but some are useful
  3. Accuracy is not always the best performance metric. In fact, in many cases (unbalanced data), it's misleading to use accuracy to evaluate model performance. [heard about 'accuracy paradox?]. Refer: https://learnerworld.tumblr.com/post/152327498485/enjoystatisticswithmebinaryclassifierperformance Choosing the right metric would depend on the objective of the problem [Do you need to minimize false positives or false negatives? I mean what's more important] Refer: https://learnerworld.tumblr.com/search/performance
  4. Tools like Excel & SPSS are not best for running, tuning & comparing different models. R & Python are great for such tasks.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.