I am trying to model a discrete choice scenario in which (i) the explanatory variables are both individual- and alternative-specific, and (ii) the number of alternatives varies between individuals. For example, in the subset below, there are three individuals; individual 1 was presented with alternatives (0,1,2) and chose alternative 1; individual 2 was presented with alternatives (0,1) and chose alternative 0; individual 3 was presented with alternatives (0,1,2,3) and chose alternative 3. IndChr1 and IndChr2 are characteristics of the individual, while AltChr1 and AltChr2 are characteristics of the alternatives presented. My response variable is the individual's choice, Chosen:

IndID Alt   Chosen IndChr1  IndChr2  AltChr1  AltChr2
1     0     0      3        5        2        3
1     1     1      3        5        4        3
1     2     0      3        5        1        2
2     0     1      1        2        3        5
2     1     0      1        2        2        3
3     0     0      4        2        3        4
3     1     0      4        2        4        3
3     2     0      4        2        5        1
3     3     1      4        2        2        2

Conditional logit models are the standard way of handling this scenario. (I'm using Python, so fortunately the recently released PyLogit package has been of great use to me). But since I am primarily interested in predictive accuracy rather than intelligibility of the model, I am interested in using Random Forests for this problem. But as I understand it, Random Forests would be analogous to standard logistic regression in that the model is fitted to the entire dataset rather than capturing the 'competitive' aspect of the alternatives for an individual (something which is captured by conditional logit models).

As I understand it, my options for modeling this scenario using Random Forests would be:

  1. Forget about modeling individuals; fit a Random Forest classifier with Chosen as the target variable and IndChr1, IndChr2, AltChr1, and AltChr2 as the explanatory variables. The predicted 'choice' for each individual would be the alternative with the highest predicted probability for that individual.

My concern with approach (1) is that, similar to using a standard logistic regression, the comparative aspect of the alternatives is lost. (An alternative may be improbable when compared against the entire dataset, but may be highly probable in comparison with the other alternatives presented for an individual).

  1. Convert the data to 'wide' format, with one record per individual. The response variable would become a multi-level categorical variable representing the chosen alternative, and each alternative would get a set of explanatory variables (e.g. Alt1_Chr1, Alt1_Chr2, Alt2_Chr1, Alt2_Chr2 etc).

My concerns with approach (2) are that it is going to be technically infeasible (my actual dataset has over 200,000 individuals, between 5 and 20 alternatives for each individual, and over 150+ explanatory variables); and also the varying choice set is going to be a problem.

My question is: is there a Random Forests-type algorithm out there that is well-suited to this kind of discrete choice problem, similar to conditional logit? Or is there another approach that I'm unaware of?

Your Answer


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.