Questions tagged [generalized-linear-model]

A generalization of linear regression allowing for nonlinear relationships via a "link function" and for the variance of the response to depend on the predicted value. (Not to be confused with "general linear model" which extends the ordinary linear model to general covariance structure and multivariate response.)

Filter by
Sorted by
Tagged with
389 votes
12 answers
385k views

Difference between logit and probit models

What is the difference between Logit and Probit model? I'm more interested here in knowing when to use logistic regression, and when to use Probit. If there is any literature which defines it using ...
Beta's user avatar
  • 6,314
26 votes
1 answer
13k views

Goodness of fit and which model to choose linear regression or Poisson

I need some advice regarding two main dilemmas in my research, which is a case study of 3 big pharmaceuticals and innovation. Number of patents per year is the dependent variable. My questions are ...
Nitzan's user avatar
  • 341
39 votes
1 answer
16k views

How do you deal with "nested" variables in a regression model?

Consider a statistical problem where you have a response variable that you want to describe conditional on an explanatory ...
Ben's user avatar
  • 125k
67 votes
4 answers
58k views

Regression for an outcome (ratio or fraction) between 0 and 1

I am thinking of building a model predicting a ratio $a/b$, where $a \le b$ and $a > 0$ and $b > 0$. So, the ratio would be between $0$ and $1$. I could use linear regression, although it doesn'...
dfrankow's user avatar
  • 3,376
58 votes
1 answer
22k views

Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?

I have built a logistic regression where the outcome variable is being cured after receiving treatment (Cure vs. No Cure). All ...
SniperBro2000's user avatar
44 votes
2 answers
26k views

Simulation of logistic regression power analysis - designed experiments

This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
B_Miner's user avatar
  • 8,610
111 votes
5 answers
57k views

Diagnostic plots for count regression

What diagnostic plots (and perhaps formal tests) do you find most informative for regressions where the outcome is a count variable? I'm especially interested in Poisson and negative binomial models, ...
half-pass's user avatar
  • 3,740
27 votes
5 answers
5k views

Do statisticians assume one can't over-water a plant, or am I just using the wrong search terms for curvilinear regression?

Almost everything I read about linear regression and GLM boils down to this: $y = f(x,\beta)$ where $f(x,\beta)$ is a non-increasing or non-decreasing function of $x$ and $\beta$ is the parameter you ...
f1r3br4nd's user avatar
  • 2,354
14 votes
2 answers
24k views

Bayesian logit model - intuitive explanation?

I must confess that I previously haven't heard of that term in any of my classes, undergrad or grad. What does it mean for a logistic regression to be Bayesian? I'm looking for an explanation with a ...
BCLC's user avatar
  • 2,414
42 votes
3 answers
66k views

Difference between generalized linear models & generalized linear mixed models

I am wondering what the differences are between mixed and unmixed GLMs. For instance, in SPSS the drop down menu allows users to fit either: ...
user9203's user avatar
  • 689
15 votes
1 answer
14k views

Interpretation of Fixed Effects from Mixed Effect Logistic Regression

I am confused by statements at a UCLA webpage about mixed effects logistic regression. They show a table of fixed effects coefficients from fitting such a model and the first paragraph belows seems to ...
B_Miner's user avatar
  • 8,610
21 votes
1 answer
10k views

Can you give a simple intuitive explanation of IRLS method to find the MLE of a GLM?

Background: I'm trying to follow Princeton's review of MLE estimation for GLM. I understand the basics of MLE estimation: likelihood, ...
ihadanny's user avatar
  • 3,290
57 votes
2 answers
60k views

How to simulate artificial data for logistic regression?

I know I'm missing something in my understanding of logistic regression, and would really appreciate any help. As far as I understand it, the logistic regression assumes that the probability of a '1' ...
zorbar's user avatar
  • 767
44 votes
2 answers
74k views

Interpretation of plot (glm.model)

Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.
Summer's user avatar
  • 441
24 votes
1 answer
24k views

What is the difference between logistic regression and Fractional response regression?

As far as I know, the difference between logistic model and fractional response model (frm) is that the dependent variable (Y) in which frm is [0,1], but logistic is {0, 1}. Further, frm uses the ...
newbie's user avatar
  • 425
95 votes
4 answers
65k views

What is the difference between a "link function" and a "canonical link function" for GLM

What's the difference between terms 'link function' and 'canonical link function'? Also, are there any (theoretical) advantages of using one over the other? For example, a binary response variable ...
steadyfish's user avatar
  • 1,922
44 votes
5 answers
141k views

How to derive the least square estimator for multiple linear regression?

In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
Saber CN's user avatar
  • 819
90 votes
5 answers
26k views

What are modern, easily used alternatives to stepwise regression?

I have a dataset with around 30 independent variables and would like to construct a generalized linear model (GLM) to explore the relationship between them and the dependent variable. I am aware that ...
fmark's user avatar
  • 4,977
64 votes
4 answers
42k views

How are regression, the t-test, and the ANOVA all versions of the general linear model?

How are they all versions of the same basic statistical method?
Amahabirsingh's user avatar
28 votes
3 answers
31k views

Interpreting plot of residuals vs. fitted values from Poisson regression

I am trying to fit data with a GLM (poisson regression) in R. When I plotted the residuals vs the fitted values, the plot created multiple (almost linear with a slight concave curve) "lines". What ...
jocelyn's user avatar
  • 281
87 votes
1 answer
105k views

How to interpret coefficients in a Poisson regression?

How can I interpret the main effects (coefficients for dummy-coded factor) in a Poisson regression? Assume the following example: ...
user avatar
75 votes
4 answers
57k views

Linear model with log-transformed response vs. generalized linear model with log link

In this paper titled "CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA" the authors write: In a generalized linear model, the mean is transformed, by the link function, instead of ...
miura's user avatar
  • 3,684
44 votes
3 answers
132k views

How to decide which glm family to use?

I have fish density data that I am trying to compare between several different collection techniques, the data has lots of zeros, and the histogram looks vaugley appropriate for a poisson distribution ...
C. Denney's user avatar
  • 695
131 votes
4 answers
134k views

When to use gamma GLMs?

The gamma distribution can take on a pretty wide range of shapes, and given the link between the mean and the variance through its two parameters, it seems suited to dealing with heteroskedasticity in ...
generic_user's user avatar
  • 13.3k
42 votes
3 answers
36k views

Interpreting residual diagnostic plots for glm models?

I am looking for guidelines on how to interpret residual plots of glm models. Especially poisson, negative binomial, binomial models. What can we expect from these plots when the models are "correct"...
Tal Galili's user avatar
  • 21.5k
68 votes
1 answer
30k views

Why is the square root transformation recommended for count data?

It is often recommended to take the square root when you have count data. (For some examples on CV, see @HarveyMotulsky's answer here, or @whuber's answer here.) On the other hand, when fitting a ...
gung - Reinstate Monica's user avatar
41 votes
2 answers
34k views

When is logistic regression solved in closed form?

Take $x \in \{0,1\}^d$ and $y \in \{0,1\}$ and suppose we model the task of predicting y given x using logistic regression. When can logistic regression coefficients be written in closed form? One ...
Yaroslav Bulatov's user avatar
33 votes
2 answers
31k views

Wald test in regression (OLS and GLMs): t- vs. z-distribution

I understand that the Wald test for regression coefficients is based on the following property that holds asymptotically (e.g. Wasserman (2006): All of Statistics, pages 153, 214-215): $$ \frac{(\hat{...
COOLSerdash's user avatar
  • 30.2k
28 votes
1 answer
9k views

Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?

We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
Haitao Du's user avatar
  • 36.8k
69 votes
9 answers
31k views

Advanced statistics books recommendation

There are several threads on this site for book recommendations on introductory statistics and machine learning but I am looking for a text on advanced statistics including, in order of priority: ...
23 votes
2 answers
32k views

Transforming proportion data: when arcsin square root is not enough

Is there a (stronger?) alternative to the arcsin square root transformation for percentage/proportion data? In the data set I'm working on at the moment, marked heteroscedasticity remains after I ...
Freya Harrison's user avatar
13 votes
2 answers
28k views

Checking residuals for normality in generalised linear models

This paper uses generalised linear models (both binomial and negative binomial error distributions) to analyse data. But then in the statistical analysis section of the methods, there is this ...
luciano's user avatar
  • 14.3k
6 votes
2 answers
10k views

Removing intercept from GLM for multiple factorial predictors only works for first factor in model

I am running a binomial logistic regression with a logit link function in R. My response is factorial [0/1] and I have two multilevel factorial predictors - let's call them $a$ and $b$ where $a$ has 4 ...
MiMi's user avatar
  • 73
71 votes
3 answers
220k views

Interpreting Residual and Null Deviance in GLM R

How to interpret the Null and Residual Deviance in GLM in R? Like, we say that smaller AIC is better. Is there any similar and quick interpretation for the deviances also? Null deviance: 1146.1 on ...
Anjali's user avatar
  • 1,011
54 votes
1 answer
60k views

Obtaining predicted values (Y=1 or 0) from a logistic regression model fit

Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by ...
tetragrammaton's user avatar
16 votes
2 answers
62k views

Understanding dummy (manual or automated) variable creation in GLM

If a factor variable (e.g. gender with levels M and F) is used in the glm formula, dummy variable(s) are created, and can be found in the glm model summary along with their associated coefficients (e....
Bryan's user avatar
  • 263
8 votes
2 answers
10k views

Understand Link Function in Generalized Linear Model

I am still trying to learn (may be the terminology issue) what does "link function" mean. For example, in logistic regression, we assume response variable is coming form binomial distribution. The $\...
Haitao Du's user avatar
  • 36.8k
25 votes
4 answers
31k views

Using offset in binomial model to account for increased numbers of patients

Two related questions from me. I have a data frame which contains numbers of patients in one column (range 10 - 17 patients) and 0s and 1s showing whether an incident happened that day. I'm using a ...
Chris Beeley's user avatar
  • 5,761
10 votes
2 answers
3k views

What is the justification for unsupervised discretization of continuous variables?

A number of sources suggest that there are many negative consequences of the discretization (categorization) of continuous variables prior to statistical analysis (sample of references [1]-[4] below). ...
Michael's user avatar
  • 235
9 votes
3 answers
15k views

Test logistic regression model using residual deviance and degrees of freedom

I was reading this page on Princeton.edu. They are performing a logistic regression (with R). At some point they calculate the probability of getting a residual deviance higher than the one they got ...
Remi.b's user avatar
  • 5,102
62 votes
4 answers
40k views

Choosing between LM and GLM for a log-transformed response variable

I'm trying to understand the philosophy behind using a Generalized Linear Model (GLM) vs a Linear Model (LM). I've created an example data set below where: $$\log(y) = x + \varepsilon $$ The ...
Marc in the box's user avatar
30 votes
2 answers
7k views

Why there are two different logistic loss formulation / notations?

I have seen two types of logistic loss formulations. We can easily show they are identical, the only difference is the definition of the label $y$. Formulation/notation 1, $y \in \{0, +1\}$: $$ L(y,\...
Haitao Du's user avatar
  • 36.8k
16 votes
2 answers
20k views

dispersion in summary.glm()

I conducted a glm.nb by glm1<-glm.nb(x~factor(group)) with group being a categorial and x being a metrical variable. When I try to get the summary of the ...
Renoir Pulitz's user avatar
8 votes
1 answer
13k views

Interaction in generalized linear model

I have 2 continuous variables as my predictors and the interaction between them, so 3 effects all together (when I center my predictors only the interaction is significant). I am using a binary logit ...
user20334's user avatar
2 votes
1 answer
13k views

What to do with GLM (Gamma) when residuals are not normally distributed?

Until now I have only done very basic/simple simple stats, but now I got stuck in all the literature/tips/forums ... It's about the following problem: I have the following data: ...
Lotw's user avatar
  • 123
94 votes
5 answers
103k views

What do the residuals in a logistic regression mean?

In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
russellpierce's user avatar
38 votes
1 answer
32k views

What is the difference between generalized estimating equations and GLMM?

I'm running a GEE on 3-level unbalanced data, using a logit link. How does this differ (in terms of the conclusions I can draw and the meaning of the coefficients) from a GLM with mixed effects (GLMM)...
user avatar
21 votes
2 answers
4k views

Family of GLM represents the distribution of the response variable or residuals?

I have been discussing with several lab members about this one, and we have gone to several sources but still don't quite have the answer: When we say a GLM has a family of poisson let's say are we ...
Derek Corcoran's user avatar
18 votes
1 answer
5k views

Properties of logistic regressions

We're working with some logistic regressions and we have realized that the average estimated probability always equals the proportion of ones in the sample; that is, the average of fitted values ...
Gabi Foix's user avatar
  • 181
12 votes
1 answer
5k views

How is a Poisson rate regression equal to a Poisson regression with corresponding offset term?

I do not understand the role of weights in "weighted Poisson regression". What exactly is being weighted? Is it the contribution of the observation to the log-likelihood of the model, or something ...
Alex's user avatar
  • 4,382

1
2 3 4 5
16