Questions tagged [generalized-linear-model]
A generalization of linear regression allowing for nonlinear relationships via a "link function" and for the variance of the response to depend on the predicted value. (Not to be confused with "general linear model" which extends the ordinary linear model to general covariance structure and multivariate response.)
4,674
questions
398
votes
12
answers
393k
views
Difference between logit and probit models
What is the difference between Logit and Probit model?
I'm more interested here in knowing when to use logistic regression, and when to use Probit.
If there is any literature which defines it using ...
- 6,416
133
votes
4
answers
140k
views
When to use gamma GLMs?
The gamma distribution can take on a pretty wide range of shapes, and given the link between the mean and the variance through its two parameters, it seems suited to dealing with heteroskedasticity in ...
- 13.6k
112
votes
5
answers
58k
views
Diagnostic plots for count regression
What diagnostic plots (and perhaps formal tests) do you find most informative for regressions where the outcome is a count variable?
I'm especially interested in Poisson and negative binomial models, ...
- 3,750
99
votes
4
answers
67k
views
What is the difference between a "link function" and a "canonical link function" for GLM
What's the difference between terms 'link function' and 'canonical link function'? Also, are there any (theoretical) advantages of using one over the other?
For example, a binary response variable ...
- 1,972
95
votes
5
answers
106k
views
What do the residuals in a logistic regression mean?
In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
- 18.9k
90
votes
5
answers
26k
views
What are modern, easily used alternatives to stepwise regression?
I have a dataset with around 30 independent variables and would like to construct a generalized linear model (GLM) to explore the relationship between them and the dependent variable.
I am aware that ...
- 4,997
87
votes
1
answer
108k
views
How to interpret coefficients in a Poisson regression?
How can I interpret the main effects (coefficients for dummy-coded factor) in a Poisson regression?
Assume the following example:
...
user734124
79
votes
4
answers
59k
views
Linear model with log-transformed response vs. generalized linear model with log link
In this paper titled "CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA" the authors write:
In a generalized linear model, the mean is transformed, by the link
function, instead of ...
- 3,774
71
votes
3
answers
225k
views
Interpreting Residual and Null Deviance in GLM R
How to interpret the Null and Residual Deviance in GLM in R? Like, we say that smaller AIC is better. Is there any similar and quick interpretation for the deviances also?
Null deviance: 1146.1 on ...
- 1,011
70
votes
1
answer
31k
views
Why is the square root transformation recommended for count data?
It is often recommended to take the square root when you have count data. (For some examples on CV, see @HarveyMotulsky's answer here, or @whuber's answer here.) On the other hand, when fitting a ...
69
votes
9
answers
32k
views
Advanced statistics books recommendation
There are several threads on this site for book recommendations on introductory statistics and machine learning but I am looking for a text on advanced statistics including, in order of priority: ...
67
votes
4
answers
61k
views
Regression for an outcome (ratio or fraction) between 0 and 1
I am thinking of building a model predicting a ratio $a/b$, where $a \le b$ and $a > 0$ and $b > 0$. So, the ratio would be between $0$ and $1$.
I could use linear regression, although it doesn'...
- 3,426
65
votes
2
answers
76k
views
What is quasi-binomial distribution (in the context of GLM)?
I'm hoping someone can provide an intuitive overview of what quasibinomial distribution is and what it does. I'm particularly interested in these points:
How quasibinomial differs to the binomial ...
- 14.5k
64
votes
4
answers
42k
views
How are regression, the t-test, and the ANOVA all versions of the general linear model?
How are they all versions of the same basic statistical method?
- 731
62
votes
4
answers
40k
views
Choosing between LM and GLM for a log-transformed response variable
I'm trying to understand the philosophy behind using a Generalized Linear Model (GLM) vs a Linear Model (LM). I've created an example data set below where:
$$\log(y) = x + \varepsilon $$
The ...
- 3,702
58
votes
1
answer
22k
views
Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?
I have built a logistic regression where the outcome variable is being cured after receiving treatment (Cure vs. No Cure). All ...
- 780
58
votes
3
answers
84k
views
Is there any difference between lm and glm for the gaussian family of glm?
Specifically, I want to know if there is a difference between lm(y ~ x1 + x2) and glm(y ~ x1 + x2, family=gaussian). I think ...
- 723
57
votes
1
answer
129k
views
Logistic regression: anova chi-square test vs. significance of coefficients (anova() vs summary() in R)
I have a logistic GLM model with 8 variables. I ran a chi-square test in R anova(glm.model,test='Chisq') and 2 of the variables turn out to be predictive when ...
- 573
57
votes
2
answers
61k
views
How to simulate artificial data for logistic regression?
I know I'm missing something in my understanding of logistic regression, and would really appreciate any help.
As far as I understand it, the logistic regression assumes that the probability of a '1' ...
- 767
54
votes
1
answer
60k
views
Obtaining predicted values (Y=1 or 0) from a logistic regression model fit
Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by ...
- 1,446
52
votes
3
answers
49k
views
When to use a GAM vs GLM
I realize this may be a potentially broad question, but I was wondering whether there are assumptions that indicate the use of a GAM (Generalized additive model) over a GLM (Generalized linear model)?
...
- 751
47
votes
3
answers
142k
views
How to decide which glm family to use?
I have fish density data that I am trying to compare between several different collection techniques, the data has lots of zeros, and the histogram looks vaugley appropriate for a poisson distribution ...
- 725
45
votes
5
answers
143k
views
How to derive the least square estimator for multiple linear regression?
In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
- 847
45
votes
2
answers
26k
views
Simulation of logistic regression power analysis - designed experiments
This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
- 8,780
43
votes
2
answers
76k
views
Interpretation of plot (glm.model)
Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.
- 531
43
votes
2
answers
50k
views
Purpose of the link function in generalized linear model
What is the purpose of the link function as a component of the generalized linear model? Why do we need it?
Wikipedia states:
It can be convenient to match the domain of the link function to the ...
- 1,359
43
votes
4
answers
36k
views
Interpreting residual diagnostic plots for glm models?
I am looking for guidelines on how to interpret residual plots of glm models. Especially poisson, negative binomial, binomial models. What can we expect from these plots when the models are "correct"...
- 21.8k
42
votes
3
answers
67k
views
Difference between generalized linear models & generalized linear mixed models
I am wondering what the differences are between mixed and unmixed GLMs. For instance, in SPSS the drop down menu allows users to fit either:
...
- 689
42
votes
2
answers
36k
views
When is logistic regression solved in closed form?
Take $x \in \{0,1\}^d$ and $y \in \{0,1\}$ and suppose we model the task of predicting y given x using logistic regression. When can logistic regression coefficients be written in closed form?
One ...
- 6,237
42
votes
2
answers
196k
views
Error "system is computationally singular" when running a glm
I'm using the robustbase package to run a glm estimation. However when I do it, I get the following error:
...
- 603
41
votes
1
answer
17k
views
How do you deal with "nested" variables in a regression model?
Consider a statistical problem where you have a response variable that you want to describe conditional on an explanatory ...
- 130k
40
votes
1
answer
33k
views
What is the difference between generalized estimating equations and GLMM?
I'm running a GEE on 3-level unbalanced data, using a logit link. How does this differ (in terms of the conclusions I can draw and the meaning of the coefficients) from a GLM with mixed effects (GLMM)...
user6666
39
votes
4
answers
54k
views
What is theta in a negative binomial regression fitted with R?
I've got a question concerning a negative binomial regression: Suppose that you have the following commands:
...
- 6,003
39
votes
3
answers
113k
views
How to calculate goodness of fit in glm (R)
I have the following result from running glm function.
How can I interpret the following values:
Null deviance
Residual deviance
AIC
Do they have something to do with the goodness of fit? Can I ...
- 915
37
votes
2
answers
36k
views
Diagnostics for generalized linear (mixed) models (specifically residuals)
I am currently struggling with finding the right model for difficult count data (dependent variable). I have tried various different models (mixed effects models are necessary for my kind of data) ...
- 1,184
37
votes
3
answers
5k
views
Why Beta/Dirichlet Regression are not considered Generalized Linear Models?
The premise is this quote from vignette of R package betareg1.
Further-more, the model shares some properties (such as linear
predictor, link function, ...
- 19.4k
34
votes
2
answers
31k
views
What are the assumptions of negative binomial regression?
I'm working with a large data set (confidential, so I can't share too much), and came to the conclusion a negative binomial regression would be necessary. I've never done a glm regression before, and ...
- 489
33
votes
2
answers
6k
views
Why do we model noise in linear regression but not logistic regression?
The canonical probabilistic interpretation of linear regression is that $y$ is equal to $\theta^Tx$, plus a Gaussian noise random variable $\epsilon$.
However, in standard logistic regression, we don'...
- 1,071
33
votes
2
answers
32k
views
Wald test in regression (OLS and GLMs): t- vs. z-distribution
I understand that the Wald test for regression coefficients is based on the following property that holds asymptotically (e.g. Wasserman (2006): All of Statistics, pages 153, 214-215):
$$
\frac{(\hat{...
- 31k
31
votes
4
answers
2k
views
What important ideas came since Nelder and McCullagh's book Generalized Linear Models (a 40 year old book)?
I read not too long ago Nelder and McCullagh's book Generalized Linear Models and thought the book was fantastic and I consider it a useful manual on the subject. Not surprising that's the case, ...
- 1,373
31
votes
1
answer
99k
views
How the 'NA' values are treated in glm in R
I have a data table T1, that contains nearly a thousand variables (V1) and around 200 million data points. The data is sparse and most of the entries are NA. Each datapoints have a unique id and date ...
- 969
31
votes
1
answer
15k
views
Nonlinear vs. generalized linear model: How do you refer to logistic, Poisson, etc. regression?
I have a question about semantics that I would like fellow statisticians' opinions on.
We know models such as logistic, Poisson, etc. fall under the umbrella of generalized linear models. The model ...
- 1,883
30
votes
4
answers
53k
views
Best way to deal with heteroscedasticity?
I have a plot of residual values of a linear model in function of the fitted values where the heteroscedasticity is very clear. However I'm not sure how I should proceed now because as far as I ...
- 301
30
votes
2
answers
56k
views
How to interpret parameters in GLM with family=Gamma
I have a question regarding parameter interpretation for a GLM with a gamma distributed dependent variable. This is what R returns for my GLM with a log-link:
...
Cajira
30
votes
4
answers
76k
views
Pseudo R squared formula for GLMs
I found a formula for pseudo $R^2$ in the book Extending the Linear Model with R, Julian J. Faraway (p. 59).
$$1-\frac{\text{ResidualDeviance}}{\text{NullDeviance}}$$.
Is this a common formula for ...
- 6,003
30
votes
2
answers
7k
views
Why there are two different logistic loss formulation / notations?
I have seen two types of logistic loss formulations. We can easily show they are identical, the only difference is the definition of the label $y$.
Formulation/notation 1, $y \in \{0, +1\}$:
$$
L(y,\...
- 37.3k
29
votes
2
answers
12k
views
Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares?
Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares?
It seems not clear to me because logistic loss and least squares loss are completely ...
- 37.3k
29
votes
1
answer
91k
views
Comparing levels of factors after a GLM in R
Here is a little background about my situation: my data refer to the number of prey successfully eaten by a predator. As the number of prey is limited (25 available) in each trial, I had a column &...
- 323
28
votes
3
answers
132k
views
How to deal with an error such as "Coefficients: 14 not defined because of singularities" in R?
When doing a GLM and you get the "not defined because of singularities" error in the anova output, how does one counteract this error from happening?
Some have suggested that it is due to ...
- 1,267
28
votes
1
answer
9k
views
Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?
We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
- 37.3k