2
$\begingroup$

Let's say I have a set of binary predictions along with ground truths for a binary classification problem. The predictions are made by an unknown classifier. My task is to reverse engineer the classifier such that I obtain similar predictions on the same data. How would I go about doing this? What would I need to know to make a reasonable approximation of the unknown classifier?

My intuition on how to begin solving the problem is the following:

i) Make an educated guess on what variables might have been used to train the unknown classifier.

ii) Use those variables to train a known classifier with the unknown classifier's predictions as the target variable.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I agree with the basic approach: train a set of new classifiers on the predictions of the unknown classifier. But I'm not entirely sure why you'd want to manually guess at the predictors - why not just use them all and let the classifiers figure out which ones contribute predictive power?

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your answer. I have no way of knowing which "all" predictors are since the predictors used to train the unknown classifier are unobservable. However, I am able to make an educated guess and match prospective predictor variables from an external dataset. I guess part of the answer to my question "What would I need to know to make a reasonable approximation of the unknown classifier?" would be "the predictors used". $\endgroup$
    – ProfessionalUsername
    Commented Sep 24 at 10:58
  • $\begingroup$ In fact, this sounds like a typical classification problem : you have your data, you have labels (even if they come from an "unknown classifier") that you are trying to predict $\endgroup$
    – afloy
    Commented Sep 24 at 11:40
  • $\begingroup$ @ProfessionalUsername Yes, if you don't know the set of predictors that were used, this becomes much harder. How well this approach works will depend strongly on how good your 'educated guess' about them is. $\endgroup$
    – mkt
    Commented Sep 24 at 16:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.