I've been researching Generative Models recently, and Probabilistic Graphical Models.
Every time I read about Generative Models, I see they're trying to predict $P(x,y)$ or equivalently $P(x|y)$ and $P(y)$, and I understand this. But every example uses classification problems and makes a comparison to discriminative models, which learn either $P(y|x)$ ($\propto P(x|y)P(y) = P(x,y)$ by Bayes' Rule) or even simpler just a map from $x \rightarrow y$.
In regression problems we generally just try to find that relationship $x \rightarrow y$. But shouldn't it in principle be possible to find a probability distribution over continuous output values too? Shouldn't it in principle be possible to model the whole of a continuously-valued problem with probability distributions internally? You would need to keep $P(x, y)$ as a function rather than as a table of discrete values, but I see no reason that should be impossible.
Are there learning models that do that sort of probabilistic reasoning for regression problems? Are there any that could be considered Generative which can handle the regression case?
Whenever I Google it, I just see results for Logistic Regression, which has long seemed like a misnomer to me, because LR is for classification.