1
$\begingroup$

Perhaps more logical/philosophical rather than math question. In binary classification setting, classes are most often not symmetrical and one of them is considered to be "positive" or "success", while other one is "negative" or "failure". This fact is used for classification with skewed posterior probability threshold, it defines such notions as true positives, false negatives, etc and corresponding evaluation measures.

Now the question - does the notion of success class belong to the model, i.e. once you have fitted the model based on the training features/response - success class must be defined at this point and changing it afterwards is a violation? Or one could have a fitted model and still change the notion of the success class - say for predicting new data? At this point the training data/labels are N/A, and classifier has done its job - so does it mean that the success class notion should not/ could not be changed after this point?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ I suppose, just tentatively, that it won't violation, and you don't have to re-fit the model, only to "invert" some its ouptut figures accordingly. But I suppose it so only for binary, univariate situation. If classification is multi-class or is mutivariate binary - it will be different. $\endgroup$
    – ttnphns
    Commented Aug 11, 2015 at 10:14
  • $\begingroup$ @ttnphns Not sure why you mention "univariate/multivariate" settings and how they affect this matter. Regarding the "re-fitting" - yes, success class shouldn't affect the fitting part at all... But the necessity of "inverting" something if the success is changed implies that the success class must be the part of the model - is that what you've meant? $\endgroup$
    – Oleg Shirokikh
    Commented Aug 11, 2015 at 19:06
  • 2
    $\begingroup$ I don't understand the question. Can you give a concrete use case where you would want to change the classes? The way I interpret the question, you want to make a model to distinguish, say, cats and dogs and then somehow use it to recognize apples. If so, that doesn't make a lot of sense. $\endgroup$
    – Marc Claesen
    Commented Aug 11, 2015 at 22:08
  • $\begingroup$ @MarcClaesen - I don't mean to change the class labels, only the notion of what is being considered as a success class/ case/ positive label, whatever you call it. $\endgroup$
    – Oleg Shirokikh
    Commented Aug 11, 2015 at 22:22
  • 1
    $\begingroup$ I still don't see the difference. You make a model for a given target class and then use it for something else. Intuitively, that cannot work well. $\endgroup$
    – Marc Claesen
    Commented Aug 12, 2015 at 6:13

1 Answer 1

Reset to default
2
$\begingroup$

The model does not know what "successful" and "unsuccessful" mean. All the model knows is the $0$$/$$1$ encoding you pass to it (which has an equivalence with a $\pm1$ encoding, if you prefer that). Consequently, if you change what $0$ and $1$ mean, the model is ignorant of this and keeps doing what you trained it to do.

Perhaps think of it this way: you train a regression model that makes extremely accurate predictions of distances in meters, but then you realize that your American customer wants the output in feet. Do you use the model output and tell your customer that is how many feet you predict (and be off by a factor of three-and-a-bit), or do you convert to feet?

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.