How are distributions and regression models related? This is likely a very simple question for many of you but is something that has been poorly covered in the statistics courses I've taken to date. We have talked extensively about distributions (normal, binomial, Poisson, etc.) and about regression models (linear, logistic, Poisson, cox), but the link between the two has never been properly explained to me. 
One of my slides from class states that "defining the distribution defines how the variance of the outcome variable is defined." But I'm unsure what this actually means in practice.
Could anyone explain in simplistic terms how distributions are linked to regressions, and what the implications would be of misspecifying a distribution in a generalized linear model? Perhaps using a Poisson distribution and Poisson regression as an example?
 A: Long story short, you're maximizing likelihoods of the distributions in order to estimate the parameters of the distribution which are your $\beta s$.
In a regression model inside the generalized linear model, you have have a bunch of different distributions you can possibly work with. You want to estimate their population mean $\mu$ mean as $x\beta$ where $\beta$ is a vector of coefficients/weights for each corresponding value of $x$ independent variable. 
Your data comes from some process, which you guess. So if it's count data you think it probably comes from a Poisson distribution with some unknown parameters. If it's height then you reckon that probably most people are clustered symmetrically around the mean, with a few extremes on either tail, so you say they come from a normal distribution. If it's income or reaction times then from experience you know that those things are right skewed, so maybe they come from a Gamma distribution, or an Inverse Gaussian distribution. You can even just try and use your data to guess by matching its shape to the possible shapes of the distribution, again using the skewed example, it won't match a normal distribution which must be roughly symmetrical. And so on (you just try to guess which distribution matches the shape of your data or is known to reflect the process your modelling).   
Then you take that distribution, and you maximize the probability of your data, either through differentiation eg. for the normal distribution to get regression, or through some iterative method eg. for logistic regression. Then you solve for your $\beta s$. In cox proportional hazards model, you don't use a distribution but you do maximize partial likelihoods every time there is an event, hence why it's semi parametric. And so on. 
The key is that the mean of something can be expressed as $x\beta$. In Poisson regression it's actually $\mu$ that is estimated. eg. using a simple identity link $E(Y_i)=x\beta$, but for logistic regression it's the odds or odds ratio (if it's categorical) and for the cox proportional hazards model, it's hazard ratios, and so on. There's a link function depending on the distribution eg. a log, logit, identity, or even inverse function that can be applied to $x \beta$ to get it to be in a linear form i.e. $y=\beta_1 x_1 +\beta_2 x_2 +\beta_3 x_3 +...+\epsilon$ Now they look like a normal regression, which incidentally just has a link function of identity, so you don't even have to transform it to have that nice equation. 
Mis-specifying just means you're biased. No matter how big your sample gets it's always be systematically wrong, because you're always trying to estimate the wrong thing. You're using the wrong distribution to estimate $\mu$. 
