Questions tagged [generalized-linear-model]
A generalization of linear regression allowing for nonlinear relationships via a "link function" and for the variance of the response to depend on the predicted value. (Not to be confused with "general linear model" which extends the ordinary linear model to general covariance structure and multivariate response.)
762
questions
389
votes
12
answers
385k
views
Difference between logit and probit models
What is the difference between Logit and Probit model?
I'm more interested here in knowing when to use logistic regression, and when to use Probit.
If there is any literature which defines it using ...
26
votes
1
answer
13k
views
Goodness of fit and which model to choose linear regression or Poisson
I need some advice regarding two main dilemmas in my research, which is a case study of 3 big pharmaceuticals and innovation. Number of patents per year is the dependent variable.
My questions are
...
39
votes
1
answer
16k
views
How do you deal with "nested" variables in a regression model?
Consider a statistical problem where you have a response variable that you want to describe conditional on an explanatory ...
67
votes
4
answers
58k
views
Regression for an outcome (ratio or fraction) between 0 and 1
I am thinking of building a model predicting a ratio $a/b$, where $a \le b$ and $a > 0$ and $b > 0$. So, the ratio would be between $0$ and $1$.
I could use linear regression, although it doesn'...
58
votes
1
answer
22k
views
Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?
I have built a logistic regression where the outcome variable is being cured after receiving treatment (Cure vs. No Cure). All ...
44
votes
2
answers
26k
views
Simulation of logistic regression power analysis - designed experiments
This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
111
votes
5
answers
57k
views
Diagnostic plots for count regression
What diagnostic plots (and perhaps formal tests) do you find most informative for regressions where the outcome is a count variable?
I'm especially interested in Poisson and negative binomial models, ...
27
votes
5
answers
5k
views
Do statisticians assume one can't over-water a plant, or am I just using the wrong search terms for curvilinear regression?
Almost everything I read about linear regression and GLM boils down to this: $y = f(x,\beta)$ where $f(x,\beta)$ is a non-increasing or non-decreasing function of $x$ and $\beta$ is the parameter you ...
14
votes
2
answers
24k
views
Bayesian logit model - intuitive explanation?
I must confess that I previously haven't heard of that term in any of my classes, undergrad or grad.
What does it mean for a logistic regression to be Bayesian? I'm looking for an explanation with a ...
42
votes
3
answers
66k
views
Difference between generalized linear models & generalized linear mixed models
I am wondering what the differences are between mixed and unmixed GLMs. For instance, in SPSS the drop down menu allows users to fit either:
...
15
votes
1
answer
14k
views
Interpretation of Fixed Effects from Mixed Effect Logistic Regression
I am confused by statements at a UCLA webpage about mixed effects logistic regression. They show a table of fixed effects coefficients from fitting such a model and the first paragraph belows seems to ...
21
votes
1
answer
10k
views
Can you give a simple intuitive explanation of IRLS method to find the MLE of a GLM?
Background:
I'm trying to follow Princeton's review of MLE estimation for GLM.
I understand the basics of MLE estimation: likelihood, ...
57
votes
2
answers
60k
views
How to simulate artificial data for logistic regression?
I know I'm missing something in my understanding of logistic regression, and would really appreciate any help.
As far as I understand it, the logistic regression assumes that the probability of a '1' ...
44
votes
2
answers
74k
views
Interpretation of plot (glm.model)
Can anyone tell me how to interpret the 'residuals vs fitted', 'normal q-q', 'scale-location', and 'residuals vs leverage' plots? I am fitting a binomial GLM, saving it and then plotting it.
24
votes
1
answer
24k
views
What is the difference between logistic regression and Fractional response regression?
As far as I know, the difference between logistic model and fractional response model (frm) is that the dependent variable (Y) in which frm is [0,1], but logistic is {0, 1}. Further, frm uses the ...
95
votes
4
answers
65k
views
What is the difference between a "link function" and a "canonical link function" for GLM
What's the difference between terms 'link function' and 'canonical link function'? Also, are there any (theoretical) advantages of using one over the other?
For example, a binary response variable ...
44
votes
5
answers
141k
views
How to derive the least square estimator for multiple linear regression?
In the simple linear regression case $y=\beta_0+\beta_1x$, you can derive the least square estimator $\hat\beta_1=\frac{\sum(x_i-\bar x)(y_i-\bar y)}{\sum(x_i-\bar x)^2}$ such that you don't have to ...
90
votes
5
answers
26k
views
What are modern, easily used alternatives to stepwise regression?
I have a dataset with around 30 independent variables and would like to construct a generalized linear model (GLM) to explore the relationship between them and the dependent variable.
I am aware that ...
64
votes
4
answers
42k
views
How are regression, the t-test, and the ANOVA all versions of the general linear model?
How are they all versions of the same basic statistical method?
28
votes
3
answers
31k
views
Interpreting plot of residuals vs. fitted values from Poisson regression
I am trying to fit data with a GLM (poisson regression) in R. When I plotted the residuals vs the fitted values, the plot created multiple (almost linear with a slight concave curve) "lines". What ...
87
votes
1
answer
105k
views
How to interpret coefficients in a Poisson regression?
How can I interpret the main effects (coefficients for dummy-coded factor) in a Poisson regression?
Assume the following example:
...
75
votes
4
answers
57k
views
Linear model with log-transformed response vs. generalized linear model with log link
In this paper titled "CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA" the authors write:
In a generalized linear model, the mean is transformed, by the link
function, instead of ...
44
votes
3
answers
132k
views
How to decide which glm family to use?
I have fish density data that I am trying to compare between several different collection techniques, the data has lots of zeros, and the histogram looks vaugley appropriate for a poisson distribution ...
131
votes
4
answers
134k
views
When to use gamma GLMs?
The gamma distribution can take on a pretty wide range of shapes, and given the link between the mean and the variance through its two parameters, it seems suited to dealing with heteroskedasticity in ...
42
votes
3
answers
36k
views
Interpreting residual diagnostic plots for glm models?
I am looking for guidelines on how to interpret residual plots of glm models. Especially poisson, negative binomial, binomial models. What can we expect from these plots when the models are "correct"...
68
votes
1
answer
30k
views
Why is the square root transformation recommended for count data?
It is often recommended to take the square root when you have count data. (For some examples on CV, see @HarveyMotulsky's answer here, or @whuber's answer here.) On the other hand, when fitting a ...
41
votes
2
answers
34k
views
When is logistic regression solved in closed form?
Take $x \in \{0,1\}^d$ and $y \in \{0,1\}$ and suppose we model the task of predicting y given x using logistic regression. When can logistic regression coefficients be written in closed form?
One ...
33
votes
2
answers
31k
views
Wald test in regression (OLS and GLMs): t- vs. z-distribution
I understand that the Wald test for regression coefficients is based on the following property that holds asymptotically (e.g. Wasserman (2006): All of Statistics, pages 153, 214-215):
$$
\frac{(\hat{...
28
votes
1
answer
9k
views
Is there any intuitive explanation of why logistic regression will not work for perfect separation case? And why adding regularization will fix it?
We have many good discussions about perfect separation in logistic regression. Such as, Logistic regression in R resulted in perfect separation (Hauck-Donner phenomenon). Now what? and Logistic ...
69
votes
9
answers
31k
views
Advanced statistics books recommendation
There are several threads on this site for book recommendations on introductory statistics and machine learning but I am looking for a text on advanced statistics including, in order of priority: ...
23
votes
2
answers
32k
views
Transforming proportion data: when arcsin square root is not enough
Is there a (stronger?) alternative to the arcsin square root transformation for percentage/proportion data? In the data set I'm working on at the moment, marked
heteroscedasticity remains after I ...
13
votes
2
answers
28k
views
Checking residuals for normality in generalised linear models
This paper uses generalised linear models (both binomial and negative binomial error distributions) to analyse data. But then in the statistical analysis section of the methods, there is this ...
6
votes
2
answers
10k
views
Removing intercept from GLM for multiple factorial predictors only works for first factor in model
I am running a binomial logistic regression with a logit link function in R. My response is factorial [0/1] and I have two multilevel factorial predictors - let's call them $a$ and $b$ where $a$ has 4 ...
71
votes
3
answers
220k
views
Interpreting Residual and Null Deviance in GLM R
How to interpret the Null and Residual Deviance in GLM in R? Like, we say that smaller AIC is better. Is there any similar and quick interpretation for the deviances also?
Null deviance: 1146.1 on ...
54
votes
1
answer
60k
views
Obtaining predicted values (Y=1 or 0) from a logistic regression model fit
Let's say that I have an object of class glm (corresponding to a logistic regression model) and I'd like to turn the predicted probabilities given by ...
16
votes
2
answers
62k
views
Understanding dummy (manual or automated) variable creation in GLM
If a factor variable (e.g. gender with levels M and F) is used in the glm formula, dummy variable(s) are created, and can be found in the glm model summary along with their associated coefficients (e....
8
votes
2
answers
10k
views
Understand Link Function in Generalized Linear Model
I am still trying to learn (may be the terminology issue) what does "link function" mean. For example, in logistic regression, we assume response variable is coming form binomial distribution.
The $\...
25
votes
4
answers
31k
views
Using offset in binomial model to account for increased numbers of patients
Two related questions from me. I have a data frame which contains numbers of patients in one column (range 10 - 17 patients) and 0s and 1s showing whether an incident happened that day. I'm using a ...
10
votes
2
answers
3k
views
What is the justification for unsupervised discretization of continuous variables?
A number of sources suggest that there are many negative consequences of the discretization (categorization) of continuous variables prior to statistical analysis (sample of references [1]-[4] below).
...
9
votes
3
answers
15k
views
Test logistic regression model using residual deviance and degrees of freedom
I was reading this page on Princeton.edu. They are performing a logistic regression (with R). At some point they calculate the probability of getting a residual deviance higher than the one they got ...
62
votes
4
answers
40k
views
Choosing between LM and GLM for a log-transformed response variable
I'm trying to understand the philosophy behind using a Generalized Linear Model (GLM) vs a Linear Model (LM). I've created an example data set below where:
$$\log(y) = x + \varepsilon $$
The ...
30
votes
2
answers
7k
views
Why there are two different logistic loss formulation / notations?
I have seen two types of logistic loss formulations. We can easily show they are identical, the only difference is the definition of the label $y$.
Formulation/notation 1, $y \in \{0, +1\}$:
$$
L(y,\...
16
votes
2
answers
20k
views
dispersion in summary.glm()
I conducted a glm.nb by
glm1<-glm.nb(x~factor(group))
with group being a categorial and x being a metrical variable. When I try to get the summary of the ...
8
votes
1
answer
13k
views
Interaction in generalized linear model
I have 2 continuous variables as my predictors and the interaction between them, so 3 effects all together (when I center my predictors only the interaction is significant). I am using a binary logit ...
2
votes
1
answer
13k
views
What to do with GLM (Gamma) when residuals are not normally distributed?
Until now I have only done very basic/simple simple stats, but now I got stuck in all the literature/tips/forums ... It's about the following problem:
I have the following data:
...
94
votes
5
answers
103k
views
What do the residuals in a logistic regression mean?
In answering this question John Christie suggested that the fit of logistic regression models should be assessed by evaluating the residuals. I'm familiar with how to interpret residuals in OLS, they ...
38
votes
1
answer
32k
views
What is the difference between generalized estimating equations and GLMM?
I'm running a GEE on 3-level unbalanced data, using a logit link. How does this differ (in terms of the conclusions I can draw and the meaning of the coefficients) from a GLM with mixed effects (GLMM)...
21
votes
2
answers
4k
views
Family of GLM represents the distribution of the response variable or residuals?
I have been discussing with several lab members about this one, and we have gone to several sources but still don't quite have the answer:
When we say a GLM has a family of poisson let's say are we ...
18
votes
1
answer
5k
views
Properties of logistic regressions
We're working with some logistic regressions and we have realized that the average estimated probability always equals the proportion of ones in the sample; that is, the average of fitted values ...
12
votes
1
answer
5k
views
How is a Poisson rate regression equal to a Poisson regression with corresponding offset term?
I do not understand the role of weights in "weighted Poisson regression". What exactly is being weighted? Is it the contribution of the observation to the log-likelihood of the model, or something ...