1
$\begingroup$

I recently touch the idea of Generative adversarial networks, which is a competition between a generative network and a discriminative network.

This idea makes me think of replacing the word "network" into a general machine learning model or algorithm. One thing comes to my mind is the difference between regression and classification, which seems to me that it is comparable to the difference between a generative model and a discriminative model.

I know there are many clarification questions asked on this site such as:

Why not approach classification through regression?

So this question is merely asking for confirmation or clarification of the idea:

Is it legitimate to say that regression is a data generative task and classification is a data discriminative task?

Improve this question
$\endgroup$
9
$\begingroup$

There are no discriminative or generative tasks, but discriminative and generative models, for both regression and classification. There is a very nice paper that discusses this difference: On Discriminative vs. Generative classifiers: A comprarison of logistic regression and naive Bayes.

Basically, discriminative models attempt to estimate the conditional probability $P(y|x)$, where $y$ is the output conditioned on the input $x$. Generative models on the other hand estimate $P(x, y) = P(y|x)P(x)$. This allows you to sample from the generative model (and hence the name), because you also model how samples $x$ are generated.

Improve this answer
$\endgroup$
  • $\begingroup$ So you mean there are regression tasks and classification tasks, not discriminative or generative ones. What I want to make sure of is the nature of regression task, which can be used to predict future data, is also a data generative task. $\endgroup$ – Bossliaw Jun 18 '17 at 9:58
  • 1
    $\begingroup$ "So you mean there are regression tasks and classification tasks, not discriminative or generative ones." -> yes $\endgroup$ – jpmuc Jun 18 '17 at 10:15
  • $\begingroup$ "I want to make sure of is the nature of regression task, which can be used to predict future data, is also a data generative task." yes and no. Yes, the goal is to predict data. But no, it is not generative because you can predict. Where do you get your samples from?. In a discriminative model you do not know how to generate new samples (p(x)?). But in a generative you have a model of how samples are generated (p(x)). In other words, generative models also take into account how likely is a given sample to happen. $\endgroup$ – jpmuc Jun 18 '17 at 10:23
  • $\begingroup$ So do you basically mean that in a regression task, using a discriminative model only predicts the values, however, using a generative model predicts not only the values but also the confidence of the values? $\endgroup$ – Bossliaw Jun 18 '17 at 10:47
  • $\begingroup$ @jpmuc Would you mind answering to this question? You mean a classification or regression model which is supervised can be considered as a discriminative model? $\endgroup$ – Media Jan 9 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.