Geometric interpretation of generalized linear model

Question

For linear model $y=x\beta+e$, we can have a nice geometric interpretation of estimated model via OLS: $\hat{y}=x\hat{\beta}+\hat{e}$. $\hat{y}$ is the projection of y onto the space spanned by x and residual $\hat{e}$ is perpendicular to this space spanned by x.

Now, my question is: is there any geometric interpretation of generalized linear model (logistic regression, Poission, survival)? I am very curious about how to interpret the estimated binary logistic regression model $\hat{p} = \textrm{logistic}(x\hat{\beta})$ geometrically, in a similar way as linear model. It even does not have an error term.

I found one talk about geometric Interpretation for Generalized Linear Models. http://statweb.stanford.edu/~lpekelis/talks/13_obs_studies.html#(7). Unfortunately, the figures are not available and it is quite hard to picture.

Any help, referencing, and suggestion will be greatly appreciated!!!

Answer 1

I think that you best bet is the thesis of Dongwen Luo from Massey University, On the geometry of generalized linear models; it is available online here. In particular you want to focus on Chapt. 3 - The Geometry of GLMs (and more particular in section 3.4). He employs two different "geometrical domains"; one before and one after the canonical link transformation. Some of the basic theoretical machinery stems from Fienberg's work on The Geometry of an r × c Contingency Table. As advocated in Luo's thesis:

For a sample of size $n$, $R^n$ splits into an orthogonal direct sum of the sufficiency space $S$ and the auxiliary space $A$. The MLE of the mean $\hat{\mu}$ lies in the intersection of the sufficiency affine plane $T = s + A$ and the untransformed model space $M_R$. The link transformed mean vector $g(\hat{\mu})$ lies in the transformed mean space $g(M_R)$.

Clearly both $S$ and $A$ need to be at least 2-D and $R^n = S \oplus A$. Under this theoretical framework $\hat{\mu}$ and the data vector $y$ have the same projection onto any direction in the sufficiency space.

Assuming you have differential geometry knowledge, the book of Kass and Vos Geometrical Foundations of Asymptotic Inference should provide a solid foundation on this matter. This paper on The Geometry of Asymptotic Inference is freely available from the author's website.

Finally, to answer your question whether there is "any geometric interpretation of generalized linear model (logistic regression, Poisson, survival)". Yes, there is one; and depends on the link function used. The observations themselves are viewed as a vector in that link transformed space. It goes without saying you will be looking at higher-dimensional manifolds as your sample size and/or the number of columns of your design matrix is increasing.