0
$\begingroup$

Basically, supervised machine learning classification boils down to predicting an output for a test input, given training data with input-output mappings.

I think All predictions can be covered by these:
- input seen
- input unseen
- input seen but output ambiguous

Let's deal with 2 simple examples.

1-Unseen input
training data:

input output
0 1

testing data:

input output
0 ?
1 ?

If input is 0, simple - we saw the same input in training, so predict same output - 1, (unless we feel that this data point was a rare/wrong one and we ignore the training data's recommendation and choose our own).
If input is 1, we did not see that input in training, so what should we predict ?
1 - because it is the only valid output we saw in training,
0 - because if input changed, output 'should' also change,
0/1 - coin flip (offload the responsibility to a random function)
0/1 - depending on success criteria and favoring one of the outputs when guessing (false-positive/negative etc.)
?

Adding one more input:
training data :

input1 input2 output
0 0 1
1 0 1
1 1 0

testing data:

input1 input2 output
0 1 ?

we haven't seen the exact input before, but we have seen 2 similar inputs ('0 0' and '1 1' both of which are just flipped by 1 bit), so should we predict the output as some probability/combination of similar seen outputs ?

2-Ambiguous input
training data:

input output
0 0
0 1

testing data:

input output
0 ?

input is 0, which we saw in training, but it had 2 outputs, so what should we predict ?
0 - because it is a valid output we saw in training,
1 - because it is a valid output we saw in training,
0/1 - coin-flip (offload the responsibility to a random function)
?

$\endgroup$
1
$\begingroup$

It depends on your success criteria. In case you want higher accuracy for the one of the classes, then you shouldn't classify unknown into that class. If you want better recall then probably classify all unknown to that class (but precision will suffer). And everywhere in between. In case you have many independent variables as is the case in bag of words linear model you can just ignore the new word and the variable that is related to it.

$\endgroup$
  • $\begingroup$ Could you answer my question here ? $\endgroup$ – d-_-b Mar 28 '16 at 22:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.