0
$\begingroup$

I want to predict the outcome of one data row and accuracy of the model is 50%, so if I re-predict it for 3 times in a row then what happens to the accuracy? Will it be 87.5% (50+25+12.5)?

Or will it remain 50%?

I am working with decision tree for supervised learning. I am extracting parameters from image from a sample. If I take three images from same sample and make predictions for 3 times what will be the improvement?

$\endgroup$
  • 2
    $\begingroup$ If it were so wouldn't it be really simple to achieve asymptotically increasing accuracy for any problem? $\endgroup$ – Firebug Oct 2 '16 at 21:08
  • $\begingroup$ Is there any mathematical answer to this? $\endgroup$ – chintan zaveri Oct 3 '16 at 3:59
  • 2
    $\begingroup$ What do you mean by "re-predict?" $\endgroup$ – Taylor Oct 3 '16 at 4:39
  • $\begingroup$ Predicting with slight modification, re capturing the data $\endgroup$ – chintan zaveri Oct 3 '16 at 4:41
  • 2
    $\begingroup$ If you say a wish three times, will that make it come true? $\endgroup$ – whuber Oct 3 '16 at 14:44
1
$\begingroup$

Two ways to think about this:

  1. You have an algorithm and you have a data set X. The algorithm acts on the data set to produce an output you call a "prediction". You believe, based on other similar data sets, that the algorithm makes the correct prediction 50% of the time. But every time the algorithm acts on data set X, it gives the same prediction. The accuracy of the prediction is not improved by running it multiple times against the same data set.

  2. You have a test that has some indeterminacy built into it. You know that it will make the right prediction for data set X 95% of the time, but it will make the wrong prediction 5% of the time. This is the classic Bayes setup: running the test multiple times will change the probability that the prediction, based on conditional probability, is correct.

So the answer is "it depends" :-)

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for details. I am working with decision tree for supervised learning. I am extracting parameters from image from a sample. If I take three images from same sample and make predictions for 3 times what will be the improvement? $\endgroup$ – chintan zaveri Oct 3 '16 at 14:46
0
$\begingroup$

The 87.5% calculation assumes that the outcomes of your prediction are statistically independent. But, thinking of your model as a fixed function $f$, your input as a fixed vector $x$, and your "slight modification" as an offset $\varepsilon$, it is certainly not the case that $f(x + \varepsilon_1)$ is independent of $f(x + \varepsilon_2)$, so the increase in probability of being correct is going to be tempered by the correlation between the two outputs.

Usually, machine learning predictions are smooth in some sense in most directions; this is why we can do machine learning in the first place. So, in a classification problem, $f(x + \varepsilon_1)$ is going to be the same as $f(x + \varepsilon_2)$ most of the time. If you do this 1000 times and get the same answer every time, this definitely does not mean that your answer is going to be (nearly) 100%.

On the other hand, if you do this 1000 times and get an answer of positive class 500 times and negative class 500 times, this means that your model is very uncertain about the inputs (assuming your $\varepsilon$ is on an appropriate scale), and you should therefore be less confident in your prediction for $x$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for details. I am working with decision tree for supervised learning. I am extracting parameters from image from a sample. If I take three images from same sample and make predictions for 3 times what will be the improvement? $\endgroup$ – chintan zaveri Oct 3 '16 at 20:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.