9
$\begingroup$

I found a paper saying that a Generalized linear model with an identity link function was used. They standardize some continuous independent variable as well as the continuous dependent variable and then run a GLM with identity link to analyse the main effects of the IVs on the DV and the interactions between IVs.

My question is: using a GLM with an identity link function for standardized variables isn't the same as running a simple linear regression? Why did they choose to use a GLM?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
7
$\begingroup$

For a conditional normal distribution, the result would indeed be in line with the normal linear model.

Example in R

# Normal linear model fitted by OLS
summary(lm(Sepal.Length ~ Sepal.Width, data = iris))

# Output
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   6.5262     0.4789   13.63   <2e-16 ***
Sepal.Width  -0.2234     0.1551   -1.44    0.152    

# GLM with conditional normal response and identity link
summary(glm(Sepal.Length ~ Sepal.Width, data = iris))

# Output
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   6.5262     0.4789   13.63   <2e-16 ***
Sepal.Width  -0.2234     0.1551   -1.44    0.152

For all other distributions in the GLM family (e.g. Gamma, Poisson or Bernoulli), the results would differ, e.g. by taking into account the variance heterogeneity that is implied by the distributional family and also by different numerical techniques (iteratively reweighted least-squares instead of a single least-squares iteration).

So e.g. for the Gamma:

summary(glm(Sepal.Length ~ Sepal.Width, data = iris, 
+             family = Gamma(link = "identity")))

# Output
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   6.5656     0.4792   13.70   <2e-16 ***
Sepal.Width  -0.2362     0.1544   -1.53    0.128

This is an additive model for a response with conditional Gamma distribution, correctly taking into account the non-homogeneity of the variance induced by the Gamma assumption.

While using an identity link with non-normal conditional response might lead to numerical instabilities in certain cases, it is a neat trick to e.g. adjust a difference in two proportions for confounders: to do so, you would run a logistic GLM with identity link.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for the detailed answer. So you can use an identity link even when the distribution is non-normal? I thought that you need to use a different link function based on the type of distribution. $\endgroup$
    – John Walk
    Commented Jun 11, 2020 at 9:49
  • 1
    $\begingroup$ Yes - this will yield an additive model for the given distribution. There are some disadvantages (potential numerical instability at the boundary of the support of the distribution, e.g. if predictions are close to 0 for a Gamma; slightly biased predictions if not the canonical link is used). E.g. in insurance modeling, claim heights are often modelled by a Gamma GLM with log link, which is not the canonical (inverse) link. $\endgroup$
    – Michael M
    Commented Jun 11, 2020 at 10:13
7
$\begingroup$

Without a paper it's impossible to know. It could be simply that they wrote a standard set of scripts using GLM function, and use different links, but in this case they used only identity. The main benefit is that you have smaller code base by not writing specific code for OLS and using the generic code for GLM, it's easy to try different settings in more generic code and probability of bugs is lowered too.

This happens to me sometimes. I have the scripts to run ARIMA, and run ARIMA(0,0,0) for a OLS regression too. This way you have fewer scripts, and less chance of a bug

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.