I am watching andrew ng's video lectures on machine learning. I am trying to understand what is even the point of generalized linear models. I understand what goes on step by step in deriving things related to generalized models but as a whole it just looks like a lot of much-ado-about-nothing. Here is my understanding (In reference to modeling a bernoull distribution):
- We make the assumption that the output is distributed according to a bernoulli distribution with parameter $\phi$
- $p(y;\phi) = \phi^{y}(1-\phi)^{1-y}$ $\tag 1$
- We use craft to represent the same equation such that it is in the structure of $ExponentialFamily(\eta): p(y;\eta) = b(y)(\eta^{T}T(y)-a(y)$ where $T(y)$ is a sufficient statistic
- For modeling, we assume the hypothesis to be $h(x;\theta) = E[T(y)|x]$
- We extract out the values of $a,b,T,\eta$ in terms of $\phi$. $\phi = \frac1{1+e^{-\eta}}$ and for bernoulli $T(y) = y$. We write the likelihood of data as a function of $\eta$ instead of $\phi$. We assume a linear relationship between $\eta$ and ($x,\theta$) in that $\eta = \theta^T.x$
- we train this model with our fav algorithm to maximize the likelihood
Questions:
This sounds like a lot of 'why did we go through this all?'. We went from one parameter to another and now we are maximizing likelihood over that parameter. Why?
What did the assumptions of $\eta = \theta^T.x$ and $h(x;\theta) = E[T(y)|x]$ achieve?