What is the appropriate machine learning algorithm for this problem?

Question

I am working on a problem which looks like this:

Input Variables

• Categorical

  • a
  • b
  • c
  • d

• Continuous

  • e

Output Variables

• Discrete(Integers)

  • v
  • x
  • y

• Continuous

  • z

The major issue that I am facing is that Output Variables are not totally independent of each other and there is no relation that can be established between them. That is, there is a dependence but not due to the causality (one value being high doesn't imply that the other will be high too but the chances of other being higher will improve)

An Example would be:

v - Number of Ad Impressions

x - Number of Ad Clicks

y - Number of Conversions

z - Revenue

Now, for an Ad to be clicked, it has to first appear on a search, so Click is somewhat dependent on Impression.

Again, for an Ad to be Converted, it has to be first clicked, so again Conversion is somewhat dependent on Click.

So running 4 instances of the problem predicting each of the output variables doesn't make sense to me. Infact there should be some way to predict all 4 together taking care of their implicit dependencies.

But as you can see, there won't be a direct relation, infact there would be a probability that is involved but which can't be worked out manually.

Plus the output variables are not Categorical but are in fact Discrete and Continuous.

Any inputs on how to go about solving this problem. Also guide me to existing implementations for the same and which toolkit to use to quickly implement the solution.

Just a random guess - I think this problem can be targeted by Bayesian Networks. What do you think ?

Answer 1

About any regression method can do the job from what I can see.

How good it will work depends very much on the data (Is the data too hard for a linear model? Are there too little samples for a neural network?) and your desire to understand what is going on (a neural network cannot tell you much, a Bayesian network can, but finding the structure is non trivial).

Answer 2

I can see that this problem can be modeled with Bayesian networks. First, you need to decide which dependencies are there in your problem. As you said,

(1) v depends on a,b,c,d,e

(2) x depends on a,b,c,d,e,v

(3) y depends on a,b,c,d,e,x

and finally,

(4) z depends on a,b,c,d,e,v,x,y

Second, you need to implement these distributions. For example, for (1), you need to model P(v|a,b,c,d,e) how you do this is upto you. You can train a classifier, learn a regressor, etc. You will also need to model the prior probabilities of a,b,c,d,e. At this point, you can write the full joint distribution of all your variables using the conditional probabilities you have.

Finally, you can answer any inference query with what you have. For example, let's say you're interested in answering P(z>q), the probability of revenue being higher than q. $P(Z>q) = \int_q^\infty P(z)dz$.

$P(z) = \sum_q \sum_b \sum_c \sum_d \sum_e \sum_v \sum_x \sum_y P(z,a,b,c,d,e,v,x,y)$ where the joint factors into your conditional probabilities:

$P(z,a,b,c,d,e,v,x,y) = P(z|a,b,c,d,e,v,y,x) P(v|a,b,c,d,e) \dots$

Hope this helps.