Skip to main content

Questions tagged [generalized-linear-model]

A generalization of linear regression allowing for nonlinear relationships via a "link function" and for the variance of the response to depend on the predicted value. (Not to be confused with "general linear model" which extends the ordinary linear model to general covariance structure and multivariate response.)

Filter by
Sorted by
Tagged with
395 votes
12 answers

Difference between logit and probit models

What is the difference between Logit and Probit model? I'm more interested here in knowing when to use logistic regression, and when to use Probit. If there is any literature which defines it using ...
Beta's user avatar
  • 6,386
27 votes
1 answer

Goodness of fit and which model to choose linear regression or Poisson

I need some advice regarding two main dilemmas in my research, which is a case study of 3 big pharmaceuticals and innovation. Number of patents per year is the dependent variable. My questions are ...
Nitzan's user avatar
  • 371
40 votes
1 answer

How do you deal with "nested" variables in a regression model?

Consider a statistical problem where you have a response variable that you want to describe conditional on an explanatory ...
Ben's user avatar
  • 128k
45 votes
2 answers

Simulation of logistic regression power analysis - designed experiments

This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
B_Miner's user avatar
  • 8,810
67 votes
4 answers

Regression for an outcome (ratio or fraction) between 0 and 1

I am thinking of building a model predicting a ratio $a/b$, where $a \le b$ and $a > 0$ and $b > 0$. So, the ratio would be between $0$ and $1$. I could use linear regression, although it doesn'...
dfrankow's user avatar
  • 3,386
58 votes
1 answer

Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?

I have built a logistic regression where the outcome variable is being cured after receiving treatment (Cure vs. No Cure). All ...
SniperBro2000's user avatar
112 votes
5 answers

Diagnostic plots for count regression

What diagnostic plots (and perhaps formal tests) do you find most informative for regressions where the outcome is a count variable? I'm especially interested in Poisson and negative binomial models, ...
half-pass's user avatar
  • 3,750
27 votes
5 answers

Do statisticians assume one can't over-water a plant, or am I just using the wrong search terms for curvilinear regression?

Almost everything I read about linear regression and GLM boils down to this: $y = f(x,\beta)$ where $f(x,\beta)$ is a non-increasing or non-decreasing function of $x$ and $\beta$ is the parameter you ...
f1r3br4nd's user avatar
  • 2,354
14 votes
2 answers

Bayesian logit model - intuitive explanation?

I must confess that I previously haven't heard of that term in any of my classes, undergrad or grad. What does it mean for a logistic regression to be Bayesian? I'm looking for an explanation with a ...
BCLC's user avatar
  • 2,444
42 votes
3 answers

Difference between generalized linear models & generalized linear mixed models

I am wondering what the differences are between mixed and unmixed GLMs. For instance, in SPSS the drop down menu allows users to fit either: ...
user9203's user avatar
  • 689
57 votes
2 answers

How to simulate artificial data for logistic regression?

I know I'm missing something in my understanding of logistic regression, and would really appreciate any help. As far as I understand it, the logistic regression assumes that the probability of a '1' ...
zorbar's user avatar
  • 767
15 votes
1 answer

Interpretation of Fixed Effects from Mixed Effect Logistic Regression

I am confused by statements at a UCLA webpage about mixed effects logistic regression. They show a table of fixed effects coefficients from fitting such a model and the first paragraph belows seems to ...
B_Miner's user avatar
  • 8,810
21 votes
1 answer

Can you give a simple intuitive explanation of IRLS method to find the MLE of a GLM?

Background: I'm trying to follow Princeton's review of MLE estimation for GLM. I understand the basics of MLE estimation: likelihood, ...
ihadanny's user avatar
  • 3,350
45 votes
5 answers

How to derive the least square estimator for multiple linear regression?

In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
Saber CN's user avatar
  • 849
44 votes
2 answers

Interpretation of plot (glm.model)

Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.
Summer's user avatar
  • 441
24 votes
1 answer

What is the difference between logistic regression and Fractional response regression?

As far as I know, the difference between logistic model and fractional response model (frm) is that the dependent variable (Y) in which frm is [0,1], but logistic is {0, 1}. Further, frm uses the ...
newbie's user avatar
  • 425
98 votes
4 answers

What is the difference between a "link function" and a "canonical link function" for GLM

What's the difference between terms 'link function' and 'canonical link function'? Also, are there any (theoretical) advantages of using one over the other? For example, a binary response variable ...
steadyfish's user avatar
  • 1,962
77 votes
4 answers

Linear model with log-transformed response vs. generalized linear model with log link

In this paper titled "CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA" the authors write: In a generalized linear model, the mean is transformed, by the link function, instead of ...
miura's user avatar
  • 3,734
28 votes
3 answers

Interpreting plot of residuals vs. fitted values from Poisson regression

I am trying to fit data with a GLM (poisson regression) in R. When I plotted the residuals vs the fitted values, the plot created multiple (almost linear with a slight concave curve) "lines". What ...
jocelyn's user avatar
  • 281
90 votes
5 answers

What are modern, easily used alternatives to stepwise regression?

I have a dataset with around 30 independent variables and would like to construct a generalized linear model (GLM) to explore the relationship between them and the dependent variable. I am aware that ...
fmark's user avatar
  • 4,987
64 votes
4 answers

How are regression, the t-test, and the ANOVA all versions of the general linear model?

How are they all versions of the same basic statistical method?
Amahabirsingh's user avatar
131 votes
4 answers

When to use gamma GLMs?

The gamma distribution can take on a pretty wide range of shapes, and given the link between the mean and the variance through its two parameters, it seems suited to dealing with heteroskedasticity in ...
generic_user's user avatar
  • 13.5k
87 votes
1 answer

How to interpret coefficients in a Poisson regression?

How can I interpret the main effects (coefficients for dummy-coded factor) in a Poisson regression? Assume the following example: ...
user avatar
45 votes
3 answers

How to decide which glm family to use?

I have fish density data that I am trying to compare between several different collection techniques, the data has lots of zeros, and the histogram looks vaugley appropriate for a poisson distribution ...
C. Denney's user avatar
  • 705
42 votes
3 answers

Interpreting residual diagnostic plots for glm models?

I am looking for guidelines on how to interpret residual plots of glm models. Especially poisson, negative binomial, binomial models. What can we expect from these plots when the models are "correct"...
Tal Galili's user avatar
  • 21.8k
68 votes
1 answer

Why is the square root transformation recommended for count data?

It is often recommended to take the square root when you have count data. (For some examples on CV, see @HarveyMotulsky's answer here, or @whuber's answer here.) On the other hand, when fitting a ...
gung - Reinstate Monica's user avatar
43 votes
2 answers

When is logistic regression solved in closed form?

Take $x \in \{0,1\}^d$ and $y \in \{0,1\}$ and suppose we model the task of predicting y given x using logistic regression. When can logistic regression coefficients be written in closed form? One ...
Yaroslav Bulatov's user avatar
32 votes
2 answers

Wald test in regression (OLS and GLMs): t- vs. z-distribution

I understand that the Wald test for regression coefficients is based on the following property that holds asymptotically (e.g. Wasserman (2006): All of Statistics, pages 153, 214-215): $$ \frac{(\hat{...
COOLSerdash's user avatar
69 votes
9 answers

Advanced statistics books recommendation

There are several threads on this site for book recommendations on introductory statistics and machine learning but I am looking for a text on advanced statistics including, in order of priority: ...
28 votes
1 answer

Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?

We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
Haitao Du's user avatar
  • 37.2k
23 votes
2 answers

Transforming proportion data: when arcsin square root is not enough

Is there a (stronger?) alternative to the arcsin square root transformation for percentage/proportion data? In the data set I'm working on at the moment, marked heteroscedasticity remains after I ...
Freya Harrison's user avatar
13 votes
2 answers

Checking residuals for normality in generalised linear models

This paper uses generalised linear models (both binomial and negative binomial error distributions) to analyse data. But then in the statistical analysis section of the methods, there is this ...
luciano's user avatar
  • 14.5k
6 votes
2 answers

Removing intercept from GLM for multiple factorial predictors only works for first factor in model

I am running a binomial logistic regression with a logit link function in R. My response is factorial [0/1] and I have two multilevel factorial predictors - let's call them $a$ and $b$ where $a$ has 4 ...
MiMi's user avatar
  • 73
71 votes
3 answers

Interpreting Residual and Null Deviance in GLM R

How to interpret the Null and Residual Deviance in GLM in R? Like, we say that smaller AIC is better. Is there any similar and quick interpretation for the deviances also? Null deviance: 1146.1 on ...
Anjali's user avatar
  • 1,011
54 votes
1 answer

Obtaining predicted values (Y=1 or 0) from a logistic regression model fit

Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by ...
tetragrammaton's user avatar
16 votes
2 answers

Understanding dummy (manual or automated) variable creation in GLM

If a factor variable (e.g. gender with levels M and F) is used in the glm formula, dummy variable(s) are created, and can be found in the glm model summary along with their associated coefficients (e....
Bryan's user avatar
  • 263
10 votes
2 answers

What is the justification for unsupervised discretization of continuous variables?

A number of sources suggest that there are many negative consequences of the discretization (categorization) of continuous variables prior to statistical analysis (sample of references [1]-[4] below). ...
Michael's user avatar
  • 235
8 votes
2 answers

Understand Link Function in Generalized Linear Model

I am still trying to learn (may be the terminology issue) what does "link function" mean. For example, in logistic regression, we assume response variable is coming form binomial distribution. The $\...
Haitao Du's user avatar
  • 37.2k
25 votes
4 answers

Using offset in binomial model to account for increased numbers of patients

Two related questions from me. I have a data frame which contains numbers of patients in one column (range 10 - 17 patients) and 0s and 1s showing whether an incident happened that day. I'm using a ...
Chris Beeley's user avatar
  • 5,831
16 votes
2 answers

dispersion in summary.glm()

I conducted a glm.nb by glm1<-glm.nb(x~factor(group)) with group being a categorial and x being a metrical variable. When I try to get the summary of the ...
Renoir Pulitz's user avatar
9 votes
3 answers

Test logistic regression model using residual deviance and degrees of freedom

I was reading this page on They are performing a logistic regression (with R). At some point they calculate the probability of getting a residual deviance higher than the one they got ...
Remi.b's user avatar
  • 5,152
5 votes
1 answer

What to do with GLM (Gamma) when residuals are not normally distributed?

Until now I have only done very basic/simple simple stats, but now I got stuck in all the literature/tips/forums ... It's about the following problem: I have the following data: ...
Lotw's user avatar
  • 153
94 votes
5 answers

What do the residuals in a logistic regression mean?

In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
russellpierce's user avatar
62 votes
4 answers

Choosing between LM and GLM for a log-transformed response variable

I'm trying to understand the philosophy behind using a Generalized Linear Model (GLM) vs a Linear Model (LM). I've created an example data set below where: $$\log(y) = x + \varepsilon $$ The ...
Marc in the box's user avatar
30 votes
2 answers

Why there are two different logistic loss formulation / notations?

I have seen two types of logistic loss formulations. We can easily show they are identical, the only difference is the definition of the label $y$. Formulation/notation 1, $y \in \{0, +1\}$: $$ L(y,\...
Haitao Du's user avatar
  • 37.2k
22 votes
2 answers

Family of GLM represents the distribution of the response variable or residuals?

I have been discussing with several lab members about this one, and we have gone to several sources but still don't quite have the answer: When we say a GLM has a family of poisson let's say are we ...
Derek Corcoran's user avatar
9 votes
1 answer

Interaction in generalized linear model

I have 2 continuous variables as my predictors and the interaction between them, so 3 effects all together (when I center my predictors only the interaction is significant). I am using a binary logit ...
user20334's user avatar
  • 115
40 votes
1 answer

What is the difference between generalized estimating equations and GLMM?

I'm running a GEE on 3-level unbalanced data, using a logit link. How does this differ (in terms of the conclusions I can draw and the meaning of the coefficients) from a GLM with mixed effects (GLMM)...
user avatar
18 votes
1 answer

Properties of logistic regressions

We're working with some logistic regressions and we have realized that the average estimated probability always equals the proportion of ones in the sample; that is, the average of fitted values ...
Gabi Foix's user avatar
  • 181
12 votes
1 answer

How is a Poisson rate regression equal to a Poisson regression with corresponding offset term?

I do not understand the role of weights in "weighted Poisson regression". What exactly is being weighted? Is it the contribution of the observation to the log-likelihood of the model, or something ...
Alex's user avatar
  • 4,452

2 3 4 5