As we all know , the GLM has the structure: $G(EY)=X^{T}\beta$, in which $G(.)$ is a known link function. What confuse me is that some people say that it's a semiparametric model. But in my opinion, it's a parametric model, because there is no nonparametric part and all we don't know in the previuos structure is $\beta$.

Could anybody tell me the reason why someone call it a semiparametric model. Thanks!

  • 1
    $\begingroup$ Perhaps I am missing something, but I think that if $G$ is fixed, then it is called a 'parameteric model'. In particular, if the model allows linear predictors $\beta$ and arbitrary link, then it is called 'semi-parametric model'. $\endgroup$ – user10525 May 5 '12 at 11:25
  • $\begingroup$ Maybe if one estimates the link function, it could be seen as semi-parametric. Also, could it be that "the people" mean Generalized Additive Models? $\endgroup$ – Momo May 5 '12 at 12:03
  • $\begingroup$ Semiparametric is usually used to describe a problem where some parameters are estimated but a particular unknown distribution that is part of the model does not have a specific parametric form. A good example is the Cox proportional hazard model. The regression parameters are estimated but the baseline hazard function is arbitrary and is removed from the problem by applying partial likelihood. Do you have a form of GLM that allows you to use an arbitrary error term distribution? That ia the only way I can think of to make it semiparametric. $\endgroup$ – Michael R. Chernick May 5 '12 at 12:34
  • $\begingroup$ @Michael: With "arbitrary error distribution", do you mean the error distribution of the latent variable that gets discretized? That would then correspond to using different link functions as procrastinator said. $\endgroup$ – Momo May 5 '12 at 12:42
  • 2
    $\begingroup$ Who claims that? It would help to have a reference, because in the reference there's likely to be an explanation. $\endgroup$ – jbowman May 5 '12 at 13:58

A GLM isn't a semi-parametric model, but the output from typical use of GLMs can be justified with only semi-parametric assumptions.

If one only assumes that the observations $Y_1, Y_2, ... Y_n$ are independent and that $$ g(\mathbb{E}[\,Y_i|X_i=x_i\,]) = x_i^T\beta $$ then, under mild regularity conditions, solving the equations $$ \sum_i\frac{\partial g^{-1}(x_i^T\beta)}{\partial \beta}w(g^{-1}(x_i^T\beta))(Y_i - g^{-1}(x_i^T\beta)) = \mathbf{0} $$ provides consistent estimates for parameter $\beta$. The weighting term $w$ is arbitrary, but it determines the efficiency of this approach, and the best option is to use weights inversely proportional to the variance of $Y_i$, if you know this.

How does this connect to GLMs? Well, the estimating equation above is just the score equation (i.e. the one that determines the MLE), under the assumption of a GLM. A particularly simple case of thise is when we use the "canonical" link function, chose so that part of the derivative term cancels with the inverse-variance weights, and we get $$ \sum_i x_i(Y_i - g^{-1}(x_i^T\beta)) = \mathbf{0}, $$ which should look familiar to anyone who's studied linear regression, or logistic regression, or Poisson regression.

In general, we can view the point estimates from GLMs as MLEs under a particular fully parametric model for $Y$, or as consistent & efficient estimates resulting from assumptions on only the first and second moments of $Y$ - i.e. a semi-parametric model.

Similar arguments apply to the confidence intervals these methods provide; see e.g. McCullagh and Nelder's book for the details.

  • $\begingroup$ +1. Would you agree that if $g^-{1}$ would not be treated as known but would be estimated too, this would become completely semi-parametric? $\endgroup$ – Momo May 5 '12 at 19:25
  • $\begingroup$ Well, it is completely semi-parametric, so I'm not sure what you mean. There are completely non-parametric interpretations of the point estimates, but in these the link function is still assumed; e.g. using linear regression it is assumed that one is it interested in a linear trend, summarizing the population from which the data comes. $\endgroup$ – guest May 5 '12 at 22:01
  • $\begingroup$ Because you said "A GLM isn't a semi-parametric model...". I read your answer as if you say that the model is parametric, but can also be seen as semi-parametric if you just view it as estimating mean and/or variance. With "completely semi-parametric" I meant with no parametric interpretation. It might also depend on the definition of semi- or non-parametric for that matter, because I don't see what the space $\mathcal{F}$ is in e.g. logistic regression when we see the parameter space as $\Theta \subset \mathbb{R}^k \times \mathcal{F}$. Care to help me out? $\endgroup$ – Momo May 5 '12 at 22:35
  • $\begingroup$ So when you said "this would become" initially, you meant "this model" and not "this interpretation of the output"? I was talking about the latter. If one makes the choice of $g$ very flexible, then yes, with care you can view the limit of a parametric procedure as a semi-parametric one. But it requires more caution about the regularity than it seems wise to get into here. $\endgroup$ – guest May 5 '12 at 23:27
  • 2
    $\begingroup$ My main point is that the output we get from GLMs, which its easy to justify via parametric arguments, can also be interpreted as the output from a semi-parametric analysis. This is important because the latter analysis, while numerically identical, requires many fewer assumptions. $\endgroup$ – guest May 5 '12 at 23:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.