Questions tagged [generalized-linear-model]

A generalization of linear regression allowing for nonlinear relationships via a "link function" and for the variance of the response to depend on the predicted value. (Not to be confused with "general linear model" which extends the ordinary linear model to general covariance structure and multivariate response.)

Filter by
Sorted by
Tagged with
387 votes
12 answers

Difference between logit and probit models

What is the difference between Logit and Probit model? I'm more interested here in knowing when to use logistic regression, and when to use Probit. If there is any literature which defines it using ...
Beta's user avatar
  • 6,284
130 votes
4 answers

When to use gamma GLMs?

The gamma distribution can take on a pretty wide range of shapes, and given the link between the mean and the variance through its two parameters, it seems suited to dealing with heteroskedasticity in ...
generic_user's user avatar
  • 13.2k
109 votes
5 answers

Diagnostic plots for count regression

What diagnostic plots (and perhaps formal tests) do you find most informative for regressions where the outcome is a count variable? I'm especially interested in Poisson and negative binomial models, ...
half-pass's user avatar
  • 3,691
94 votes
4 answers

What is the difference between a "link function" and a "canonical link function" for GLM

What's the difference between terms 'link function' and 'canonical link function'? Also, are there any (theoretical) advantages of using one over the other? For example, a binary response variable ...
steadyfish's user avatar
  • 1,892
93 votes
5 answers

What do the residuals in a logistic regression mean?

In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
russellpierce's user avatar
90 votes
5 answers

What are modern, easily used alternatives to stepwise regression?

I have a dataset with around 30 independent variables and would like to construct a generalized linear model (GLM) to explore the relationship between them and the dependent variable. I am aware that ...
fmark's user avatar
  • 4,957
83 votes
1 answer

How to interpret coefficients in a Poisson regression?

How can I interpret the main effects (coefficients for dummy-coded factor) in a Poisson regression? Assume the following example: ...
user avatar
74 votes
4 answers

Linear model with log-transformed response vs. generalized linear model with log link

In this paper titled "CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA" the authors write: In a generalized linear model, the mean is transformed, by the link function, instead of ...
miura's user avatar
  • 3,654
69 votes
3 answers

Interpreting Residual and Null Deviance in GLM R

How to interpret the Null and Residual Deviance in GLM in R? Like, we say that smaller AIC is better. Is there any similar and quick interpretation for the deviances also? Null deviance: 1146.1 on ...
Anjali's user avatar
  • 991
69 votes
9 answers

Advanced statistics books recommendation

There are several threads on this site for book recommendations on introductory statistics and machine learning but I am looking for a text on advanced statistics including, in order of priority: ...
68 votes
1 answer

Why is the square root transformation recommended for count data?

It is often recommended to take the square root when you have count data. (For some examples on CV, see @HarveyMotulsky's answer here, or @whuber's answer here.) On the other hand, when fitting a ...
gung - Reinstate Monica's user avatar
66 votes
4 answers

Regression for an outcome (ratio or fraction) between 0 and 1

I am thinking of building a model predicting a ratio $a/b$, where $a \le b$ and $a > 0$ and $b > 0$. So, the ratio would be between $0$ and $1$. I could use linear regression, although it doesn'...
dfrankow's user avatar
  • 3,336
63 votes
4 answers

How are regression, the t-test, and the ANOVA all versions of the general linear model?

How are they all versions of the same basic statistical method?
Amahabirsingh's user avatar
62 votes
3 answers

What is quasi-binomial distribution (in the context of GLM)?

I'm hoping someone can provide an intuitive overview of what quasibinomial distribution is and what it does. I'm particularly interested in these points: How quasibinomial differs to the binomial ...
luciano's user avatar
  • 13.9k
62 votes
4 answers

Choosing between LM and GLM for a log-transformed response variable

I'm trying to understand the philosophy behind using a Generalized Linear Model (GLM) vs a Linear Model (LM). I've created an example data set below where: $$\log(y) = x + \varepsilon $$ The ...
Marc in the box's user avatar
57 votes
3 answers

Is there any difference between lm and glm for the gaussian family of glm?

Specifically, I want to know if there is a difference between lm(y ~ x1 + x2) and glm(y ~ x1 + x2, family=gaussian). I think ...
user3682457's user avatar
57 votes
2 answers

How to simulate artificial data for logistic regression?

I know I'm missing something in my understanding of logistic regression, and would really appreciate any help. As far as I understand it, the logistic regression assumes that the probability of a '1' ...
zorbar's user avatar
  • 767
56 votes
1 answer

Logistic regression: anova chi-square test vs. significance of coefficients (anova() vs summary() in R)

I have a logistic GLM model with 8 variables. I ran a chi-square test in R anova(glm.model,test='Chisq') and 2 of the variables turn out to be predictive when ...
StreetHawk's user avatar
55 votes
1 answer

Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?

I have built a logistic regression where the outcome variable is being cured after receiving treatment (Cure vs. No Cure). All ...
SniperBro2000's user avatar
53 votes
1 answer

Obtaining predicted values (Y=1 or 0) from a logistic regression model fit

Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by ...
tetragrammaton's user avatar
47 votes
3 answers

When to use a GAM vs GLM

I realize this may be a potentially broad question, but I was wondering whether there are assumptions that indicate the use of a GAM (Generalized additive model) over a GLM (Generalized linear model)? ...
mluerig's user avatar
  • 691
44 votes
2 answers

Interpretation of plot (glm.model)

Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.
Summer's user avatar
  • 441
44 votes
3 answers

How to decide which glm family to use?

I have fish density data that I am trying to compare between several different collection techniques, the data has lots of zeros, and the histogram looks vaugley appropriate for a poisson distribution ...
C. Denney's user avatar
  • 695
43 votes
5 answers

How to derive the least square estimator for multiple linear regression?

In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
Saber CN's user avatar
  • 809
43 votes
2 answers

Simulation of logistic regression power analysis - designed experiments

This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
B_Miner's user avatar
  • 8,170
42 votes
3 answers

Difference between generalized linear models & generalized linear mixed models

I am wondering what the differences are between mixed and unmixed GLMs. For instance, in SPSS the drop down menu allows users to fit either: ...
user9203's user avatar
  • 689
42 votes
2 answers

Purpose of the link function in generalized linear model

What is the purpose of the link function as a component of the generalized linear model? Why do we need it? Wikipedia states: It can be convenient to match the domain of the link function to the ...
Chris's user avatar
  • 1,339
42 votes
2 answers

Error "system is computationally singular" when running a glm

I'm using the robustbase package to run a glm estimation. However when I do it, I get the following error: ...
NK1's user avatar
  • 603
42 votes
3 answers

Interpreting residual diagnostic plots for glm models?

I am looking for guidelines on how to interpret residual plots of glm models. Especially poisson, negative binomial, binomial models. What can we expect from these plots when the models are "correct"...
Tal Galili's user avatar
  • 21.1k
39 votes
3 answers

How to calculate goodness of fit in glm (R)

I have the following result from running glm function. How can I interpret the following values: Null deviance Residual deviance AIC Do they have something to do with the goodness of fit? Can I ...
learner's user avatar
  • 915
38 votes
1 answer

What is the difference between generalized estimating equations and GLMM?

I'm running a GEE on 3-level unbalanced data, using a logit link. How does this differ (in terms of the conclusions I can draw and the meaning of the coefficients) from a GLM with mixed effects (GLMM)...
user avatar
37 votes
2 answers

When is logistic regression solved in closed form?

Take $x \in \{0,1\}^d$ and $y \in \{0,1\}$ and suppose we model the task of predicting y given x using logistic regression. When can logistic regression coefficients be written in closed form? One ...
Yaroslav Bulatov's user avatar
37 votes
3 answers

Why Beta/Dirichlet Regression are not considered Generalized Linear Models?

The premise is this quote from vignette of R package betareg1. Further-more, the model shares some properties (such as linear predictor, link function, ...
Firebug's user avatar
  • 18.4k
36 votes
2 answers

Diagnostics for generalized linear (mixed) models (specifically residuals)

I am currently struggling with finding the right model for difficult count data (dependent variable). I have tried various different models (mixed effects models are necessary for my kind of data) ...
fsociety's user avatar
  • 1,144
35 votes
3 answers

What is theta in a negative binomial regression fitted with R?

I've got a question concerning a negative binomial regression: Suppose that you have the following commands: ...
MarkDollar's user avatar
  • 5,805
35 votes
1 answer

How do you deal with "nested" variables in a regression model?

Consider a statistical problem where you have a response variable that you want to describe conditional on an explanatory ...
Ben's user avatar
  • 118k
34 votes
2 answers

What are the assumptions of negative binomial regression?

I'm working with a large data set (confidential, so I can't share too much), and came to the conclusion a negative binomial regression would be necessary. I've never done a glm regression before, and ...
Carly's user avatar
  • 489
32 votes
2 answers

Wald test in regression (OLS and GLMs): t- vs. z-distribution

I understand that the Wald test for regression coefficients is based on the following property that holds asymptotically (e.g. Wasserman (2006): All of Statistics, pages 153, 214-215): $$ \frac{(\hat{...
COOLSerdash's user avatar
  • 29.4k
31 votes
1 answer

Nonlinear vs. generalized linear model: How do you refer to logistic, Poisson, etc. regression?

I have a question about semantics that I would like fellow statisticians' opinions on. We know models such as logistic, Poisson, etc. fall under the umbrella of generalized linear models. The model ...
Meg's user avatar
  • 1,823
30 votes
4 answers

What important ideas came since Nelder and McCullagh's book Generalized Linear Models (a 40 year old book)?

I read not too long ago Nelder and McCullagh's book Generalized Linear Models and thought the book was fantastic and I consider it a useful manual on the subject. Not surprising that's the case, ...
cgmil's user avatar
  • 1,313
30 votes
4 answers

Pseudo R squared formula for GLMs

I found a formula for pseudo $R^2$ in the book Extending the Linear Model with R, Julian J. Faraway (p. 59). $$1-\frac{\text{ResidualDeviance}}{\text{NullDeviance}}$$. Is this a common formula for ...
MarkDollar's user avatar
  • 5,805
30 votes
2 answers

Why do we model noise in linear regression but not logistic regression?

The canonical probabilistic interpretation of linear regression is that $y$ is equal to $\theta^Tx$, plus a Gaussian noise random variable $\epsilon$. However, in standard logistic regression, we don'...
kennysong's user avatar
  • 1,021
30 votes
2 answers

Why there are two different logistic loss formulation / notations?

I have seen two types of logistic loss formulations. We can easily show they are identical, the only difference is the definition of the label $y$. Formulation/notation 1, $y \in \{0, +1\}$: $$ L(y,\...
Haitao Du's user avatar
  • 36.1k
29 votes
1 answer

Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares?

Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares? It seems not clear to me because logistic loss and least squares loss are completely ...
Haitao Du's user avatar
  • 36.1k
29 votes
4 answers

Best way to deal with heteroscedasticity?

I have a plot of residual values of a linear model in function of the fitted values where the heteroscedasticity is very clear. However I'm not sure how I should proceed now because as far as I ...
TristanDM's user avatar
  • 291
29 votes
1 answer

How the 'NA' values are treated in glm in R

I have a data table T1, that contains nearly a thousand variables (V1) and around 200 million data points. The data is sparse and most of the entries are NA. Each datapoints have a unique id and date ...
user1140126's user avatar
29 votes
1 answer

Comparing levels of factors after a GLM in R

Here is a little background about my situation: my data refer to the number of prey successfully eaten by a predator. As the number of prey is limited (25 available) in each trial, I had a column &...
Anne's user avatar
  • 323
28 votes
1 answer

Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?

We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
Haitao Du's user avatar
  • 36.1k
28 votes
3 answers

Interpreting plot of residuals vs. fitted values from Poisson regression

I am trying to fit data with a GLM (poisson regression) in R. When I plotted the residuals vs the fitted values, the plot created multiple (almost linear with a slight concave curve) "lines". What ...
jocelyn's user avatar
  • 281
27 votes
5 answers

Do statisticians assume one can't over-water a plant, or am I just using the wrong search terms for curvilinear regression?

Almost everything I read about linear regression and GLM boils down to this: $y = f(x,\beta)$ where $f(x,\beta)$ is a non-increasing or non-decreasing function of $x$ and $\beta$ is the parameter you ...
f1r3br4nd's user avatar
  • 2,284

2 3 4 5