GLMs: What features are required for valid link functions and statistical distributions? Theory question here:
For Generalized Linear Models, what features do we require for a valid link function and a valid statistical distribution. I feel like the tendency of using exponential-family distributions is largely a factor of convenience (since when using log-likelihood, the logarithm and the exponent just cancel each other out), but would it be possible to use something like a Cauchy distribution?
I feel like you could be a bit perverse and choose a rectangle distribution, though you'll just have the issue that your parameters $\beta$ might not be uniquely defined.
As for the link function, I can think of one to three required properties:


*

*The function's range must cover the domain of the data.

*The function must be invertible (maybe? Or could you combine this with something like MARS?).

*The function should be continuous (seems like you'd want this, but I don't know why).


Are there any other important things we need? Integrability? Differentiability seems like it would be handy but I don't know if it's necessary.
 A: You're right that we often over-rely on exponential distributions to formulate GLMs. A link function is only half of a GLM. Also specified is a variance structure. For exponential families, the variance and link function have a mathematical relationship which gives a very simple estimating equation. However, there is often very good reason to step beyond the bounds of "regular likelihoods" and look to GLMs as estimating a meaningful relationship. 
An example of that is relative risk regression which uses a binomial variance structure but a log link to compare the relative rates of an outcome. Here, unlike logistic regression, an estimated rate ratio of 2 can be interpreted as "twice as likely to experience an outcome of interest" (an Odds ratio of 2 has no such interpretation).
I think the only consideration would be that the estimating equation (whose solution produces our estimate $\beta).
$$S(\beta) = D^TV^{-1} (Y - g(X\beta))$$
Should be asymptotically linear at it's root. I think this is required to ensure that the estimated variance is in some sense correct. Choosing $g^{-1}$ differentiable and invertible helps to ensure this, but it is not a necessary condition. 
The paper by Fahrmeir discusses this.
You can, of course, get a broader class of estimators from minimax estimation!
