When modelling a GAM model using
mgcv in R, we need to define the
family = . I tried some families (e.g., Gaussian, Gamma), R seems to build them all successfully. Does there some guildlines about how to choose the appropriate "family"?
When modelling a GAM model using
6$\begingroup$ You have to think about the distribution of the outcome conditioned on the covariates. So if you're modelling weight as a function of height, what is the distribution of weight for all 6 foot people? That will determine the family. $\endgroup$– Demetri PananosMar 18, 2019 at 3:59
1$\begingroup$ @DemetriPananos I use "fitdistrplus" package ("descdist" function) to examine the most possible distribution of the response variable. Is this method appropriate? I think it does not take "... conditioned on the covariates" into account. $\endgroup$– T XMar 20, 2019 at 11:40
2$\begingroup$ Using fitdistplus is not the appropriate method. When you use fitdistplus, you are fitting the marginal distribution of the outcome (that is the distribution of the outcome without considering covariates). If you're modelling weight as a function of height and sex, consider the distribution of weight for all 6 foot tall men. That is the distribution of the outcome conditioned on covariates. $\endgroup$– Demetri PananosMar 20, 2019 at 15:03
Here is an example of what I mean by "outcome conditioned on the covariate".
I want to do a linear regression. I have a continuous outcome and I am regressing it on a binary variable. This is equivalent to a t-test, but let's pretend we don't know that.
What most people do is look at the marginal distribution of the data. This is equivalent to plotting histogram of the outcome variable. Let's look at that now
Ew, gross, this is bimodal. Linear regression assumes the outcome is normally distributed, right? We can't use linear regression on this!
...or can we? Here is the output of a linear model fit the this data.
Call: lm(formula = y ~ x, data = d) Residuals: Min 1Q Median 3Q Max -7.3821 -1.7504 -0.0194 1.7190 7.8183 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 9.8994 0.1111 89.13 <2e-16 *** x 12.0931 0.1588 76.14 <2e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 2.511 on 998 degrees of freedom Multiple R-squared: 0.8531, Adjusted R-squared: 0.853 F-statistic: 5797 on 1 and 998 DF, p-value: < 2.2e-16
An incredibly good fit. So what gives?
The plot above is the marginal outcome. Regression, be it linear or otherwise, only cares about the conditional outcome; the distribution of the outcome conditioned on the covariates. Let's see what happens when I color the observations by the binary variable.
You can see here that the data conditioned on the outcome are normal, and hence fit into linear regression's assumptions.
So when I say "think about the outcome conditioned on covariates" what I am really asking you to do is to think about a particular set of covariates and think about the distribution of outcomes from those covariates. That will determine the family.
1$\begingroup$ Now I understand your meaning. One "practical" question: how to think about the conditional distribution of Y? For example, if there're many covariates (X1, X2, ..., X10), Is there a way to figure out the distribution of Y given X? I noticed that in "stats.stackexchange.com/questions/190763/…", Tim said "If you are dealing with continuous non-negative outcome, then you could consider the Gamma distribution, or Inverse Gaussian distribution." This seems not consider the "X1, ..., X10", because he recommend the distribution when we see non-negative Y. $\endgroup$– T XMar 22, 2019 at 10:42
$\begingroup$ You have to use some of your background knowledge on the problem. There is no way to determine from the data the most appropriate family. $\endgroup$ Mar 22, 2019 at 17:18
$\begingroup$ I'm a bit of a novice myself but this confuses me a lot...why do you say that linear regression assumes a normal distribution? I mean, you seem to be suggesting that we are happy because the data conditioned on the outcome is normal. Why is that important for linear regression? I never quite know why statisticians say that fitting a model actually assumes something about the data. Isn't the model just a model (although how well it performs will of course depend on the characteristics of the data, but that's not relying on an 'assumption') $\endgroup$– T_MJul 10, 2020 at 10:46
$\begingroup$ @T_M The inferences from the model are made under the assumption that the data are normal. The validity of those inferences hinges on that assumption. $\endgroup$ Jul 10, 2020 at 15:53
$\begingroup$ @DemetriPananos thanks for your very clear answer. Would you be kind enough to provide the data/Rscript you've used ? $\endgroup$ Mar 7, 2021 at 10:29