# elementary question: modelling the mean and generalized linear models

I am learning generalized linear models, and I there I suddeling find expressions like "modelling the mean". Also on this site, I find the sentence: "Instead of modelling the mean as was done for simple linear regression, we now model a transformation of the mean so instead of saying μi= ..."

When learning ordinary lm the course I follow never spoke of "modelling the mean", always dealt with a response and its Yi. Now suddenly the course changes terminology, speaks of modelling the mean, and speeds away without any explanation except for mumbling quickly "beware. in glms we do not deal with the response but with the mean".

I think that I realize very well that in a certain way fitting a regression line through the Yi-s we minimizes residuals relative to the line, that the line has residuals relative to the mean of Y, the mean of X is the fulcrum... Etc. but the very clear expression "modelling the mean" and the fact that I struggle to understand the explanation of glm in my course makes me suspect that I might have some gaps, at least in perspective, and should address them to really understand glms.

• punctual meaning (if any) of the expression "modelling the mean", in ordinary/simple lm and in glm
• conceptual or perspective points I may be missing in my current state that I might describe as "you have Yi-s ... fit a line ... the mean is (just) the center of mass, the fulcrum ..."
• differences between the modeling done in simple lm and the modeling done in glm (maybe this is equivalent to asking for a full explanation of glm?)
• (this question might be based on total misuderstanding on my side) how does glm "build" "the rest" of the model after having modelled/built the mean? (ie. if it were a software program that first produces the mean, what is the control flow after that step if any?)

The first two questions are vital, the others would be very useful

Linear regression models are usually modeling the mean. I.e. if we are fitting a regression using least squares, this will actually be the same as using the man in many cases, and in others, it can still be related to some "hypothetical" mean.

Let us take a simple regression model, which describes our data as $y_i=\beta_0+\beta_1*x_i+\epsilon_i$ with $\epsilon_i \sim N(0,\sigma)$ (this can easily be extended to multiple predictors). Now our data that we measured is just a sample from the full joint distribution of $Y$ and $X$. Now let us assume, we have some kind of process, that would allow us to specify some value for $x$ and repeat the draw an arbitrary number of times. What we are actually interested in is the value of $\beta_0$ and $\beta_1$, such that the means of the $y$s we can calculate from the equation go towards the same number as the means of the $y$s we are getting from our sampling process. These $\beta_0$ and $\beta_1$ are the actual values for the effect sizes we would like to know. And these are based on the mean of all draws for $Y$ and the same value $x$ as the number of draws goes towards infinity.

Now in practice this process does not work, as we cannot draw an infinite number of samples. Furthermore, the predictor $X$ may not be controllable, so that we may never even see two samples that have the exact same value for $X$. Assume for example we are predicting height based on the age of a person. It is unlikely that our sample will contain two people of the same age up to the second of birth, so that the "mean across same values of $X$" is not even something we can use or calculate. Still we want to predict the values that we would get, if we had the hypothetical sampling process above, and these values are based on the mean. Therefore, we are still modeling the mean of some hypothetical process, but we now have to estimate that value.

A lot of the confusion arises from the fact, that we can talk about multiple means here. There is the actual population mean (or rather expected value), which is the value of the mean for an infinite amount of samples. This is the mean we are interested in and this is also the mean any linear model is modeling. Now since we cannot get this from our data, we have to estimate it. It turns out, that the sample mean is an estimator for the population mean. Actually not just any estimator, but rather the best estimator we can get based on the data. Therefore, we are not really interested in the mean of the sample in itself, but only in so far, as this allows us to estimate some other hypothetical figure, that we cannot get in any other way.

In GLM we are still modeling the mean, just not of the original values $y_i$, but rather of some transformation of the $y_i$. For example we could model the mean of $log(y_i)$. This does not necessarily mean, that we are actually calculating the mean of all $log(y_i)$ given the same value for $X$ (in lm we also may have never calculated the mean for $y_i$ given the same value for $X$, especially since we may not even be able to calculate this figure as we have never seen the same value for $X$ twice). But we are still basing our modeling on some hypothetical population mean of $log(y_i)$ for the same value for $X$, which we only can estimate based on our sample.

Recall that at a high level we are trying to model a response $Y$ to predictors $x_i$. A generalized linear model consists of predictors $$\eta=\beta_0+\beta_1x_1+\dots+\beta_px_p.$$

In this model, the mean of the observation $Y$ depends on the variables $x_i$. For example, for a linear model the mean of $Y$ is determined by $\eta$, which in turn depends on the $x_i$.

The link function $g$ describes how this mean depends on possibly more complicated function of the linear predictor $\eta$. In particular, $g^{-1}(\eta)=\mu$.

In a linear model the link function is the identity and the underlying distribution is assumed to be Gaussian.