Questions tagged [generalized-linear-model]
A generalization of linear regression allowing for nonlinear relationships via a "link function" and for the variance of the response to depend on the predicted value. (Not to be confused with "general linear model" which extends the ordinary linear model to general covariance structure and multivariate response.)
4,503
questions
387
votes
12
answers
379k
views
Difference between logit and probit models
What is the difference between Logit and Probit model?
I'm more interested here in knowing when to use logistic regression, and when to use Probit.
If there is any literature which defines it using ...
- 6,284
130
votes
4
answers
130k
views
When to use gamma GLMs?
The gamma distribution can take on a pretty wide range of shapes, and given the link between the mean and the variance through its two parameters, it seems suited to dealing with heteroskedasticity in ...
- 13.2k
109
votes
5
answers
56k
views
Diagnostic plots for count regression
What diagnostic plots (and perhaps formal tests) do you find most informative for regressions where the outcome is a count variable?
I'm especially interested in Poisson and negative binomial models, ...
- 3,691
94
votes
4
answers
64k
views
What is the difference between a "link function" and a "canonical link function" for GLM
What's the difference between terms 'link function' and 'canonical link function'? Also, are there any (theoretical) advantages of using one over the other?
For example, a binary response variable ...
- 1,892
93
votes
5
answers
100k
views
What do the residuals in a logistic regression mean?
In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
- 18.1k
90
votes
5
answers
25k
views
What are modern, easily used alternatives to stepwise regression?
I have a dataset with around 30 independent variables and would like to construct a generalized linear model (GLM) to explore the relationship between them and the dependent variable.
I am aware that ...
- 4,957
83
votes
1
answer
103k
views
How to interpret coefficients in a Poisson regression?
How can I interpret the main effects (coefficients for dummy-coded factor) in a Poisson regression?
Assume the following example:
...
user734124
74
votes
4
answers
55k
views
Linear model with log-transformed response vs. generalized linear model with log link
In this paper titled "CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA" the authors write:
In a generalized linear model, the mean is transformed, by the link
function, instead of ...
- 3,654
69
votes
3
answers
215k
views
Interpreting Residual and Null Deviance in GLM R
How to interpret the Null and Residual Deviance in GLM in R? Like, we say that smaller AIC is better. Is there any similar and quick interpretation for the deviances also?
Null deviance: 1146.1 on ...
- 991
69
votes
9
answers
30k
views
Advanced statistics books recommendation
There are several threads on this site for book recommendations on introductory statistics and machine learning but I am looking for a text on advanced statistics including, in order of priority: ...
68
votes
1
answer
30k
views
Why is the square root transformation recommended for count data?
It is often recommended to take the square root when you have count data. (For some examples on CV, see @HarveyMotulsky's answer here, or @whuber's answer here.) On the other hand, when fitting a ...
66
votes
4
answers
55k
views
Regression for an outcome (ratio or fraction) between 0 and 1
I am thinking of building a model predicting a ratio $a/b$, where $a \le b$ and $a > 0$ and $b > 0$. So, the ratio would be between $0$ and $1$.
I could use linear regression, although it doesn'...
- 3,336
63
votes
4
answers
41k
views
How are regression, the t-test, and the ANOVA all versions of the general linear model?
How are they all versions of the same basic statistical method?
- 711
62
votes
3
answers
70k
views
What is quasi-binomial distribution (in the context of GLM)?
I'm hoping someone can provide an intuitive overview of what quasibinomial distribution is and what it does. I'm particularly interested in these points:
How quasibinomial differs to the binomial ...
- 13.9k
62
votes
4
answers
39k
views
Choosing between LM and GLM for a log-transformed response variable
I'm trying to understand the philosophy behind using a Generalized Linear Model (GLM) vs a Linear Model (LM). I've created an example data set below where:
$$\log(y) = x + \varepsilon $$
The ...
- 3,702
57
votes
3
answers
82k
views
Is there any difference between lm and glm for the gaussian family of glm?
Specifically, I want to know if there is a difference between lm(y ~ x1 + x2) and glm(y ~ x1 + x2, family=gaussian). I think ...
- 703
57
votes
2
answers
59k
views
How to simulate artificial data for logistic regression?
I know I'm missing something in my understanding of logistic regression, and would really appreciate any help.
As far as I understand it, the logistic regression assumes that the probability of a '1' ...
- 767
56
votes
1
answer
123k
views
Logistic regression: anova chi-square test vs. significance of coefficients (anova() vs summary() in R)
I have a logistic GLM model with 8 variables. I ran a chi-square test in R anova(glm.model,test='Chisq') and 2 of the variables turn out to be predictive when ...
- 563
55
votes
1
answer
21k
views
Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?
I have built a logistic regression where the outcome variable is being cured after receiving treatment (Cure vs. No Cure). All ...
- 750
53
votes
1
answer
60k
views
Obtaining predicted values (Y=1 or 0) from a logistic regression model fit
Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by ...
- 1,436
47
votes
3
answers
43k
views
When to use a GAM vs GLM
I realize this may be a potentially broad question, but I was wondering whether there are assumptions that indicate the use of a GAM (Generalized additive model) over a GLM (Generalized linear model)?
...
- 691
44
votes
2
answers
72k
views
Interpretation of plot (glm.model)
Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.
- 441
44
votes
3
answers
125k
views
How to decide which glm family to use?
I have fish density data that I am trying to compare between several different collection techniques, the data has lots of zeros, and the histogram looks vaugley appropriate for a poisson distribution ...
- 695
43
votes
5
answers
138k
views
How to derive the least square estimator for multiple linear regression?
In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
- 809
43
votes
2
answers
25k
views
Simulation of logistic regression power analysis - designed experiments
This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
- 8,170
42
votes
3
answers
65k
views
Difference between generalized linear models & generalized linear mixed models
I am wondering what the differences are between mixed and unmixed GLMs. For instance, in SPSS the drop down menu allows users to fit either:
...
- 689
42
votes
2
answers
49k
views
Purpose of the link function in generalized linear model
What is the purpose of the link function as a component of the generalized linear model? Why do we need it?
Wikipedia states:
It can be convenient to match the domain of the link function to the ...
- 1,339
42
votes
2
answers
184k
views
Error "system is computationally singular" when running a glm
I'm using the robustbase package to run a glm estimation. However when I do it, I get the following error:
...
- 603
42
votes
3
answers
35k
views
Interpreting residual diagnostic plots for glm models?
I am looking for guidelines on how to interpret residual plots of glm models. Especially poisson, negative binomial, binomial models. What can we expect from these plots when the models are "correct"...
- 21.1k
39
votes
3
answers
109k
views
How to calculate goodness of fit in glm (R)
I have the following result from running glm function.
How can I interpret the following values:
Null deviance
Residual deviance
AIC
Do they have something to do with the goodness of fit? Can I ...
- 915
38
votes
1
answer
31k
views
What is the difference between generalized estimating equations and GLMM?
I'm running a GEE on 3-level unbalanced data, using a logit link. How does this differ (in terms of the conclusions I can draw and the meaning of the coefficients) from a GLM with mixed effects (GLMM)...
user6666
37
votes
2
answers
33k
views
When is logistic regression solved in closed form?
Take $x \in \{0,1\}^d$ and $y \in \{0,1\}$ and suppose we model the task of predicting y given x using logistic regression. When can logistic regression coefficients be written in closed form?
One ...
- 6,229
37
votes
3
answers
5k
views
Why Beta/Dirichlet Regression are not considered Generalized Linear Models?
The premise is this quote from vignette of R package betareg1.
Further-more, the model shares some properties (such as linear
predictor, link function, ...
- 18.4k
36
votes
2
answers
33k
views
Diagnostics for generalized linear (mixed) models (specifically residuals)
I am currently struggling with finding the right model for difficult count data (dependent variable). I have tried various different models (mixed effects models are necessary for my kind of data) ...
- 1,144
35
votes
3
answers
50k
views
What is theta in a negative binomial regression fitted with R?
I've got a question concerning a negative binomial regression: Suppose that you have the following commands:
...
- 5,805
35
votes
1
answer
15k
views
How do you deal with "nested" variables in a regression model?
Consider a statistical problem where you have a response variable that you want to describe conditional on an explanatory ...
- 118k
34
votes
2
answers
29k
views
What are the assumptions of negative binomial regression?
I'm working with a large data set (confidential, so I can't share too much), and came to the conclusion a negative binomial regression would be necessary. I've never done a glm regression before, and ...
- 489
32
votes
2
answers
31k
views
Wald test in regression (OLS and GLMs): t- vs. z-distribution
I understand that the Wald test for regression coefficients is based on the following property that holds asymptotically (e.g. Wasserman (2006): All of Statistics, pages 153, 214-215):
$$
\frac{(\hat{...
- 29.4k
31
votes
1
answer
14k
views
Nonlinear vs. generalized linear model: How do you refer to logistic, Poisson, etc. regression?
I have a question about semantics that I would like fellow statisticians' opinions on.
We know models such as logistic, Poisson, etc. fall under the umbrella of generalized linear models. The model ...
- 1,823
30
votes
4
answers
2k
views
What important ideas came since Nelder and McCullagh's book Generalized Linear Models (a 40 year old book)?
I read not too long ago Nelder and McCullagh's book Generalized Linear Models and thought the book was fantastic and I consider it a useful manual on the subject. Not surprising that's the case, ...
- 1,313
30
votes
4
answers
74k
views
Pseudo R squared formula for GLMs
I found a formula for pseudo $R^2$ in the book Extending the Linear Model with R, Julian J. Faraway (p. 59).
$$1-\frac{\text{ResidualDeviance}}{\text{NullDeviance}}$$.
Is this a common formula for ...
- 5,805
30
votes
2
answers
6k
views
Why do we model noise in linear regression but not logistic regression?
The canonical probabilistic interpretation of linear regression is that $y$ is equal to $\theta^Tx$, plus a Gaussian noise random variable $\epsilon$.
However, in standard logistic regression, we don'...
- 1,021
30
votes
2
answers
7k
views
Why there are two different logistic loss formulation / notations?
I have seen two types of logistic loss formulations. We can easily show they are identical, the only difference is the definition of the label $y$.
Formulation/notation 1, $y \in \{0, +1\}$:
$$
L(y,\...
- 36.1k
29
votes
1
answer
11k
views
Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares?
Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares?
It seems not clear to me because logistic loss and least squares loss are completely ...
- 36.1k
29
votes
4
answers
49k
views
Best way to deal with heteroscedasticity?
I have a plot of residual values of a linear model in function of the fitted values where the heteroscedasticity is very clear. However I'm not sure how I should proceed now because as far as I ...
- 291
29
votes
1
answer
95k
views
How the 'NA' values are treated in glm in R
I have a data table T1, that contains nearly a thousand variables (V1) and around 200 million data points. The data is sparse and most of the entries are NA. Each datapoints have a unique id and date ...
- 929
29
votes
1
answer
87k
views
Comparing levels of factors after a GLM in R
Here is a little background about my situation: my data refer to the number of prey successfully eaten by a predator. As the number of prey is limited (25 available) in each trial, I had a column &...
- 323
28
votes
1
answer
9k
views
Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?
We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
- 36.1k
28
votes
3
answers
31k
views
Interpreting plot of residuals vs. fitted values from Poisson regression
I am trying to fit data with a GLM (poisson regression) in R. When I plotted the residuals vs the fitted values, the plot created multiple (almost linear with a slight concave curve) "lines". What ...
- 281
27
votes
5
answers
5k
views
Do statisticians assume one can't over-water a plant, or am I just using the wrong search terms for curvilinear regression?
Almost everything I read about linear regression and GLM boils down to this: $y = f(x,\beta)$ where $f(x,\beta)$ is a non-increasing or non-decreasing function of $x$ and $\beta$ is the parameter you ...
- 2,284