I want to develop a predictive model based on a binary dependent variable (1 if default, 0 otherwise). Based on methods like logistic regression, decision trees, and random forest in R.

My independent variables consist of:

  • Multiple binary variables
  • One categorical variable, which contains 10 different categories. This variable is split to 10 new dummy variables, where 9 is included in the model.
  • Other variables are integer numbers e.g., year, etc. Q1: When I develop these models in R, what type of structure should the variables be? I have seen that in decision trees and random forest, many use “factor” for the classification dependent variable. Is this correct for binary? And what should be used for the independent variables?

Q2: When evaluating the models, which performance metric is the best choice to use in this case? Accuracy, AUC, F1 score, etc.

  • $\begingroup$ Welcome to Cross Validated! The R-specific question is off-topic here and is appropriate for Stack Overflow. // Your performance metric question is on-topic here and has been answered many times. Log loss, using the predicted probabilities (not threshold-based categorization) is the "gold standard", as Frank Harrell puts it. $\endgroup$ – Dave 2 days ago
  • $\begingroup$ @Dave, log loss is another name for cross entropy? $\endgroup$ – Single Malt 2 days ago
  • $\begingroup$ @SingleMalt I forget if they have differences like scaling factors (like MSE vs SSE), but they are equivalent as loss functions in that they have the same argmin. $\endgroup$ – Dave 2 days ago
  • $\begingroup$ That makes sense and the analogy with MSE vs SSE is very close. Another good aspect of cross-entropy is that its formula is intuitive, and furthermore the equivalent formulation in terms of entropy plus KL divergence (also known as relative entropy here) is also intuitive. $\endgroup$ – Single Malt 2 days ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.