7
$\begingroup$

When modelling a GAM model using mgcv in R, we need to define the family = . I tried some families (e.g., Gaussian, Gamma), R seems to build them all successfully. Does there some guildlines about how to choose the appropriate "family"?

$\endgroup$
3
  • 5
    $\begingroup$ You have to think about the distribution of the outcome conditioned on the covariates. So if you're modelling weight as a function of height, what is the distribution of weight for all 6 foot people? That will determine the family. $\endgroup$ Mar 18 '19 at 3:59
  • 1
    $\begingroup$ @DemetriPananos I use "fitdistrplus" package ("descdist" function) to examine the most possible distribution of the response variable. Is this method appropriate? I think it does not take "... conditioned on the covariates" into account. $\endgroup$
    – T X
    Mar 20 '19 at 11:40
  • 1
    $\begingroup$ Using fitdistplus is not the appropriate method. When you use fitdistplus, you are fitting the marginal distribution of the outcome (that is the distribution of the outcome without considering covariates). If you're modelling weight as a function of height and sex, consider the distribution of weight for all 6 foot tall men. That is the distribution of the outcome conditioned on covariates. $\endgroup$ Mar 20 '19 at 15:03
17
$\begingroup$

Here is an example of what I mean by "outcome conditioned on the covariate".

I want to do a linear regression. I have a continuous outcome and I am regressing it on a binary variable. This is equivalent to a t-test, but let's pretend we don't know that.

What most people do is look at the marginal distribution of the data. This is equivalent to plotting histogram of the outcome variable. Let's look at that now

enter image description here

Ew, gross, this is bimodal. Linear regression assumes the outcome is normally distributed, right? We can't use linear regression on this!

...or can we? Here is the output of a linear model fit the this data.

Call:
lm(formula = y ~ x, data = d)

Residuals:
    Min      1Q  Median      3Q     Max 
-7.3821 -1.7504 -0.0194  1.7190  7.8183 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   9.8994     0.1111   89.13   <2e-16 ***
x            12.0931     0.1588   76.14   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.511 on 998 degrees of freedom
Multiple R-squared:  0.8531,    Adjusted R-squared:  0.853 
F-statistic:  5797 on 1 and 998 DF,  p-value: < 2.2e-16

An incredibly good fit. So what gives?

The plot above is the marginal outcome. Regression, be it linear or otherwise, only cares about the conditional outcome; the distribution of the outcome conditioned on the covariates. Let's see what happens when I color the observations by the binary variable.

enter image description here

You can see here that the data conditioned on the outcome are normal, and hence fit into linear regression's assumptions.

So when I say "think about the outcome conditioned on covariates" what I am really asking you to do is to think about a particular set of covariates and think about the distribution of outcomes from those covariates. That will determine the family.

$\endgroup$
5
  • 1
    $\begingroup$ Now I understand your meaning. One "practical" question: how to think about the conditional distribution of Y? For example, if there're many covariates (X1, X2, ..., X10), Is there a way to figure out the distribution of Y given X? I noticed that in "stats.stackexchange.com/questions/190763/…", Tim said "If you are dealing with continuous non-negative outcome, then you could consider the Gamma distribution, or Inverse Gaussian distribution." This seems not consider the "X1, ..., X10", because he recommend the distribution when we see non-negative Y. $\endgroup$
    – T X
    Mar 22 '19 at 10:42
  • $\begingroup$ You have to use some of your background knowledge on the problem. There is no way to determine from the data the most appropriate family. $\endgroup$ Mar 22 '19 at 17:18
  • $\begingroup$ I'm a bit of a novice myself but this confuses me a lot...why do you say that linear regression assumes a normal distribution? I mean, you seem to be suggesting that we are happy because the data conditioned on the outcome is normal. Why is that important for linear regression? I never quite know why statisticians say that fitting a model actually assumes something about the data. Isn't the model just a model (although how well it performs will of course depend on the characteristics of the data, but that's not relying on an 'assumption') $\endgroup$
    – T_M
    Jul 10 '20 at 10:46
  • $\begingroup$ @T_M The inferences from the model are made under the assumption that the data are normal. The validity of those inferences hinges on that assumption. $\endgroup$ Jul 10 '20 at 15:53
  • $\begingroup$ @DemetriPananos thanks for your very clear answer. Would you be kind enough to provide the data/Rscript you've used ? $\endgroup$ Mar 7 at 10:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.