Suppose I am building a predictive model where I am trying to predict multiple events (for instance, both the roll of a die and the toss of a coin). Most algorithms that I am familiar with work with only one target, so I'm wondering if there is a standard approach to this sort of thing.

I see two possible options. Perhaps the most naive approach would be to simply treat them as two different problems, and then combine the results. However, that has serious drawbacks when the two targets are not independent (and in many instances they might be very dependent).

A more sensible approach to me would be to make a combined target attribute. So in the case of a die and coin, we would have $6\cdot 2=12$ states ($(1, H), (1, T), (2, H)$, etc). However, this can lead to the number of states/classes in the composite target getting rather large rather quickly (what if we had 2 dice, etc.). Furthermore, this seems strange in the case that one attribute is categorical while the other is numeric (for instance if predicting temperature and type of precipitation).

Is there any standard approach to this sort of thing? Alternatively, are there any learning algorithms designed specifically to handle this?

  • $\begingroup$ Do you mean very dependent at the end of your 2nd paragraph. If so, have you thought of some type of Markov Chain approach once the first variable is estimated? $\endgroup$ – Michelle Jan 28 '12 at 6:28
  • $\begingroup$ Oops, I did indeed mean dependent and fixed it, thanks. I hadn't considered a Markov Chain approach and I'll have to think if that makes sense here; thanks. $\endgroup$ – Michael McGowan Jan 28 '12 at 7:01

This is known in the Machine Learning community as "Multi-Label Learning". There are various approaches to the problem, including the ones you describe in your question. Some resources to get you started:


Where you have two variables with the same predictors, and variable B also has variable A as a predictor, you are possibly looking at an optimization problem, where you want to optimize the estimates of A and B simultaneously. It makes no sense to optimize one, if you then get a bad estimate for the second.

This would be an operations research problem, and unfortunately outside my realm of expertise.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.