I'm quite new to all this and am just trying to understand the concepts of GLM - getting very confused between distribution assumptions and the actual regression function. Let's keep it simple with one explanatory variable.
Looking at e.g. Poisson:
Poisson regression assumes the response variable Y has a Poisson distribution, and assumes the logarithm of its expected value can be modeled by a linear combination of unknown parameters.
Now for a start, by "variable Y has a Poisson distribution" do we just mean the distribution of $\epsilon_i$ rather than $Y_i$? Clearly a line $y=exp(x)$ looks nothing like a Poisson distribution, and I think that's where my confusion is coming from.
I just don't see why a linear relationship needs to be modelled using a normal distribution, and why an exponential relationship needs to be modelled using a Poisson distribution. Why can't we just stick with the normal distribution?
Also, Wikipedia suggests the Poisson regression as a typical use for "count of occurrences in fixed amount of time/space". Again, if I want to regress something that looks like $y=exp(x)$, where's the relationship to the Poisson distribution and why is count a typical occurence?
I really hope my thoughts make sense to someone - please put some order into my thoughts!