In lecture today, my professor mentioned in the context of GDA and Generative learning, we would like to learn the joint probability $P(x, y)$, where $x \in \mathbb{R}^n$ and $y \in \{+1, -1\}$. Simply, we apply Bayes rule and realize that the key is to model $P(x \mid y) \cdot P(y)$, where clearly, $P(y)$ is the class prior. My question is that he mentioned that if $ x \mid y \sim \mathit{ExponentialFamily}(\theta_y)$, is a distribution in the exponential family, then it follows that $P(Y = y \mid x)$ is a sigmoid/logistic function. That seems intuitively clear for me based on the example provided in lecture, but does not seem algebraically obvious. How can I show this formally?
1
-
$\begingroup$ Any thoughts? I'm still not quite sure how to take care of this... $\endgroup$– Reese PathakApr 14, 2016 at 20:13
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You will need to have some information about the parametric form of $P(y)$ in order to determine the form of the posterior. He's saying that if the likelihood is from the exponential family, then the posterior is logistic, but that requires knowing the form of the prior. I don't think it follows necessarily, and he was most likely talking about logistic regression for binomial classification with some abused or confused terminology.
