GLMs: What features are required for valid link functions and statistical distributions?

Question

Theory question here:

For Generalized Linear Models, what features do we require for a valid link function and a valid statistical distribution. I feel like the tendency of using exponential-family distributions is largely a factor of convenience (since when using log-likelihood, the logarithm and the exponent just cancel each other out), but would it be possible to use something like a Cauchy distribution?

I feel like you could be a bit perverse and choose a rectangle distribution, though you'll just have the issue that your parameters $\beta$ might not be uniquely defined.

As for the link function, I can think of one to three required properties:

• The function's range must cover the domain of the data.
• The function must be invertible (maybe? Or could you combine this with something like MARS?).
• The function should be continuous (seems like you'd want this, but I don't know why).

Are there any other important things we need? Integrability? Differentiability seems like it would be handy but I don't know if it's necessary.

Answer 1

You're right that we often over-rely on exponential distributions to formulate GLMs. A link function is only half of a GLM. Also specified is a variance structure. For exponential families, the variance and link function have a mathematical relationship which gives a very simple estimating equation. However, there is often very good reason to step beyond the bounds of "regular likelihoods" and look to GLMs as estimating a meaningful relationship.

An example of that is relative risk regression which uses a binomial variance structure but a log link to compare the relative rates of an outcome. Here, unlike logistic regression, an estimated rate ratio of 2 can be interpreted as "twice as likely to experience an outcome of interest" (an Odds ratio of 2 has no such interpretation).

I think the only consideration would be that the estimating equation (whose solution produces our estimate $\beta).

$$S(\beta) = D^TV^{-1} (Y - g(X\beta))$$

Should be asymptotically linear at it's root. I think this is required to ensure that the estimated variance is in some sense correct. Choosing $g^{-1}$ differentiable and invertible helps to ensure this, but it is not a necessary condition.

The paper by Fahrmeir discusses this.

You can, of course, get a broader class of estimators from minimax estimation!