Recently there has been discussion about the distinction between classification and prediction. In particular it concerns the case where outcome $Y\sim Bern(p)$. Usually $p$ depends on covariates but we can ignore that here. The claim is that a prediction must be an estimate $\hat{p}$ of the probability $P(Y=1)$, and a classification, $\delta(\hat{p})$, is not a prediction, but instead a decision (where $\delta(x)=1$ if $x>t$ and zero otherwise).

But because $Y $ is binary, shouldn’t a prediction for $ Y $ be binary? Or, another way, an estimate of $p$ is an estimate of a parameter - in this case it is an estimate of the mean of the Bernoulli distribution. An estimate of $p$ is not an estimate of the random variable $Y.$ By definition prediction is a guess at random variable while estimation is a guess at a parameter. So $\hat{p}$ is not a prediction of $Y,$ it is an estimate of $p$, and $\delta(\hat{p})$ is not a classification/decision, it is a prediction.

  • 2
    $\begingroup$ Your question might become clearer if accompanied by a definition of what you mean by "prediction," because this term has various meanings in statistics. $\endgroup$ – whuber Jan 20 at 18:38
  • $\begingroup$ @whuber added, it is the definition you gave in the linked answer. $\endgroup$ – user0 Jan 20 at 23:49
  • $\begingroup$ You don't use it in the sense I gave, because in general a parameter is not a probability. $\endgroup$ – whuber Jan 21 at 0:25
  • $\begingroup$ @whuber updated - here I am referring to the Bernoulli case $\endgroup$ – user0 Feb 15 at 0:58
  • 1
    $\begingroup$ Thank you. I am reminded of the colloquial sense of prediction, such as what the weatherman announces. When they say "there's a 60% chance of rain tomorrow," isn't that generally understood as a prediction? Of course if someone says "it will rain tomorrow," that's a prediction, too, but it is no longer probabilistic. Accordingly, in statistics a "prediction" is generally thought to be a characterization of an unobserved random variable based on inferred relationships with observed random variables, in contradistinction to a definite statement about a value of the unobserved variable. $\endgroup$ – whuber Feb 15 at 15:23

Some nomenclature issues here. A classification is a prediction (I predict case x is class A based on the data/model).

But I think what you are getting at is the fact that most implementations of classification in computer software can return a binary classification - but this is simply a categorization based on computed probabilities relative to some threshold. You are free to change that threshold and thus your classification based on those extracted probabilities.

  • $\begingroup$ The main claim is usually that a classification is binary (or categorical) by definition and that a binary value is not prediction. I am not sure if this is equivalent to saying that a binary value is not a prediction. $\endgroup$ – user0 Jan 20 at 23:53
  • 1
    $\begingroup$ @user0 yea the inclusion or exclusion of ‘a’ is key here. $\endgroup$ – HEITZ Jan 20 at 23:56
  • $\begingroup$ @user0 I hear you but you are taking issue with semantics. I don’t think this deserves a downvote. The point I am making is valid $\endgroup$ – HEITZ Mar 13 at 7:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.