1
$\begingroup$

I have a set of 5 binary inputs and for each there are a set of 5 binary outputs. I would like to know what technique one could use to find the rule between them..? I've used Machine Learning in then past where the output (Response) is a single 1, or 0 in this case a simple classifier does the trick. I have never done this however when we have multiple outputs..

I1  I2  I3  I4  I5  O1  O2  O3  O4  O5
1    1   0   0   1  0    0  1    1   0

etc etc.... For maybe 100 rows

Any help would be appreciated. Paul.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Suppose you have a n-inputs 1-output useful predictor that predicts $P(Y=1|X)$. Logistic regression or random forest for example.

One possibility to implement it for a p-output $Y=(Y_1,Y_2,...Y_p)$ is the following:

  • Train predictor of $P(Y_1=1|X)$
  • Train predictor of $P(Y_2=1|X,Y_1)$
  • ...
  • Train predictor of $P(Y_p=1|X,Y_1,Y_2...Y_{p-1})$

Now you want to estimate $P(Y=(y_1,y_2...y_p)|X)$. Just use conditional probabilities:

$P(Y=(y_1,y_2...y_p)|X)=P(Y=y_1|X)P(Y_2=y_2|X,y_1)... P(Y_p=y_p|X,y_1,...,y_{p-1})$

This is just a possibility. The ordering is somehow arbitrary. Efficiency will obviously depend on the predictor you use. This could give better results than mere multinomial regression on all $2^p$ bins of $Y$, since a certain underlying independence of the correlation of the outputs with each other is assumed.

Improve this answer
$\endgroup$
0
$\begingroup$

Specifically, for what you are asking, there's absolutely no need for ML.

There are only 2^5=32 different combinations for the inputs. Assuming that the system is memoryless, it is easy to go through all of the inputs and see the function for each output.

Basically, you will get a table having 32 rows and 5 columns (there are 5 outputs).

Cheers.

Improve this answer
$\endgroup$
5
  • $\begingroup$ OK so it turns out that I have 8 inputs and 8 outputs...there are 255 unique inputs, with only 81 unique outputs. So a given input can have multiple same outputs. $\endgroup$
    – PaulB.
    Commented Oct 13, 2017 at 10:55
  • $\begingroup$ Well, the table size is still very small (255 rows,8 columns), so my original answer is still good. Just as a sidenote: ML is nothing more than a set of tools. Use it when those set of tools is appropriate :) $\endgroup$
    – Yoni Keren
    Commented Oct 13, 2017 at 10:59
  • $\begingroup$ Yeah I have the table (255 rows with 8 cols) but I'm sure how I 'learn' or 'see the function' of the underlying the rules..? $\endgroup$
    – PaulB.
    Commented Oct 13, 2017 at 11:27
  • $\begingroup$ @Yoni: Even 32 cells is a lot when you have 100 observations. And there are far more in this example. 32x5 would only learn marginal conditional distributions of the output, not the joint conditional distribution. The key idea of machine learning is to reduce the scope of possible distributions so that you can predict something better than with raw counting. Classically, defining a (realistic) model + regularisation makes prediction much better especially with few observations. Other methods with no explicit model (eg Random forests) achieve the same goal without explicitly defining the model. $\endgroup$
    – Benoit Sanchez
    Commented Oct 13, 2017 at 11:59
  • $\begingroup$ @PaulB unlike most binary input-single output binary functions (such as And,Xor,Nor etc), AFAIK most 5 bits input functions don't actually have a name (except specialized cases for which there exists a parallel counterpart for a 2 bits input 1 bit output). So,what do you mean but "learn" or "see the function"? A table such the one which you have created is legit for both to my mind. $\endgroup$
    – Yoni Keren
    Commented Oct 13, 2017 at 18:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.