Skip to main content

New answers tagged

4 votes

Why GLM don't have an error term and why shouldn't residuals be i.i.d?

GLMs specify a conditional distribution at each combination of predictors. When that conditional distribution is normal with constant variance, then the errors about the mean will be i.i.d. normal and ...
Glen_b's user avatar
  • 284k
2 votes

Why GLM don't have an error term and why shouldn't residuals be i.i.d?

GLMs are not supposed to generalize $y = X\beta + \varepsilon$. They generalize the equivalent$^{\dagger}$ notion of $\mathbb E\left[Y\vert X=x\right] = X\beta$. There is no error term in this about ...
Dave's user avatar
  • 63.5k
2 votes

Why GLM don't have an error term and why shouldn't residuals be i.i.d?

Arguably, regression concerns modeling the expected response. If we focus on this aspect of modeling, the "error" is less something intrinsic to a model than it is a way of calculating the ...
AdamO's user avatar
  • 63k
1 vote

Why GLM don't have an error term and why shouldn't residuals be i.i.d?

GLMs (generalized linear models) include a lot of different types of models. It's more of a family than a single model. Notably, the typical linear model is a special case of the GLM. A GLM ...
gung - Reinstate Monica's user avatar
1 vote
Accepted

One sided likelihood ratio test for a logistic regression model?

A one-sided LRT is straightforward in R using the signed LRT statistic. Fit the logistic regression models with and without the $\beta$ term. Compute the ordinary LRT statistic from the deviance ...
Gordon Smyth's user avatar
6 votes
Accepted

VGAMs/VGLMs vs Multiple GAMs/GLMs

VGAMs are an extremely flexible model class. Yes, they can be used to model data where the response is actually a vector of responses where each vector gets its own linear predictor. This is not the ...
Gavin Simpson's user avatar
4 votes

Logistic regression with labels corrupted by known noise model

I'm going to take a Bayesian approach because honestly I'm not sure how to tackle this as a Frequentist. Let's first write out the likelihood for such a model. The likelihood of $y_i$ depends on a ...
Demetri Pananos's user avatar
1 vote

Logistic regression with labels corrupted by known noise model

(Started as a comment but got too long) Some obvious references are: Label-noise robust logistic regression and its applications (2012) by Bootkrajang & Kaban and Learning From Noisy Labels By ...
usεr11852's user avatar
  • 44.4k
2 votes
Accepted

Hypothesis tests in summary of glm() are always Wald tests?

They are both Wald tests. The column is labelled as t when a variance parameter is estimated and as z when no variance ...
Thomas Lumley's user avatar
1 vote

simplified form for CDF of inverse Gaussian function, as function of the drift parameter

The probability of passage can be expressed with $\Phi$ the CDF of the normal distribution $$\begin{array}{} P(T>t) &=&\Phi\left(\sqrt{\frac{\lambda}{t}} \left(\frac{t}{\mu}-1\right)\right) ...
Sextus Empiricus's user avatar
6 votes

Why is there little difference in glm fit using poisson and gaussian family for Poisson data?

@Dave's answer is a good one (and OP seems to be satisfied with it), but for anyone else finding this question later I'll offer a different perspective. The TL;DR is that if you think of fitting a GLM ...
ischmidt20's user avatar
12 votes
Accepted

Why is there little difference in glm fit using poisson and gaussian family for Poisson data?

You’ve got models to two different data sets. For the Poisson regression, your true conditional expected values (lambda values) are given by $\exp(0.43x+0.2)$. For the linear regression, your true ...
Dave's user avatar
  • 63.5k
4 votes

In a GLM model with a gamma log link, how to interpret a negative coefficent of a dummy variable with a continuous response?

This question is similar and has an accepted answer. But, briefly, what is $e^0$? It's 1. If you had some binary variable with a $\beta$ of 0, then it is associated with zero difference in the mean of ...
Weiwen Ng's user avatar
  • 1,478
1 vote

DHARMa residual vs. predicted outliners

Nicholas answer is correct. Some more quotes from the DHARMa vignette: Note that outliers in DHARMa are values that are by default defined as values outside the simulation envelope, not in terms of a ...
Florian Hartig's user avatar
4 votes
Accepted

Why is the canonical link of a GLM with Gamma distribution the reciprocal?

Your formulation of the Gamma distribution is: $$ f(y;a,\lambda) = \frac{\lambda^a e^{-\lambda y} y^{a-1}}{\Gamma(a)} $$ where $a$ (often referred to as $k$ in some texts) is the shape parameter, and $...
Robert Long's user avatar
  • 61.6k
1 vote

Reference Request: Generalized Linear Models

My all time favourite book that introduces GLMs is Fahrmeier: https://link.springer.com/book/10.1007/978-3-642-34333-9 It starts off with the basics of linear models, but the section on GLMs should be ...
MrMidnight's user avatar
2 votes

Calculating relative importance of predictors in a poisson glm model

A very general index of relative importance that reduces to relative $R^2$ in linear models is obtained by mapping the model to a linear model and using relative $R^2$. This method also easily allows ...
Frank Harrell's user avatar
-1 votes

Calculating relative importance of predictors in a poisson glm model

You can use the glmm.hp package in R. https://academic.oup.com/jpe/article/15/6/1302/6845730 https://academic.oup.com/jpe/article/16/6/rtad038/7444979 ...
Janson's user avatar
  • 1
1 vote
Accepted

describing binomial data in likelihood models

The two approaches model different situations. In the first case, we do not observe the order of the values $0$ and $1$. All we know are the totals: how many zeroes and ones we observed, and that's ...
jbowman's user avatar
  • 39.6k
0 votes
Accepted

How to show that a stable discrete stochastic process converges to a stationary process?

For $k \in \mathbb{N}$ the covariance of $x_k$ is given by $\mathbb{E}\left[ x_k x_k^\top \right] - \mathbb{E}\left[ x_k \right] \mathbb{E}\left[ x_k \right]^\top$ where $$\mathbb{E}\left[ x_k x_k^\...
msantama's user avatar
  • 116
2 votes
Accepted

GLMs and their conditional expectation and variance

I have always wondered about this myself as well. There are proofs for it like using the score in the other answer, or the moment generating function like mentioned in the comments. But they are ...
Sextus Empiricus's user avatar
9 votes

GLMs and their conditional expectation and variance

You have to keep in mind that $Y_i, $ the responses, follow exponential family of distributions. Then we can get our required formula working with the log-likelihood of $\theta, ~\ell(\theta)$ and its ...
User1865345's user avatar
  • 8,832
2 votes

ZIP interpretation

Briefly, zero-inflated Poisson regression assumes that zeros are created by two processes: i) "true" or excess zeros and ii) sampling zeros (from the Poisson distribution). As an example (...
COOLSerdash's user avatar
  • 30.7k
4 votes
Accepted

Interpretation of generalized linear mixed-effects models: How should I proceed with a significant interaction term?

I have some comments before looking at the treatment effect comparison between "old" and "new". This is a linear mixed model (a linear regression with random effects), not a ...
dipetkov's user avatar
  • 10.2k
3 votes
Accepted

Exact equations for predictions in the hurdle model?

Root of the problem: You are right about most details but the predict(..., type = "zero") does not give the zero hurdle probability $\pi$ for the hurdle ...
Achim Zeileis's user avatar
0 votes

How does lmList calculate AIC?

I have discovered that AIC for the multispecies list model is the sum of the AICs for the separate models. Is there anything else we need to know?
stweb's user avatar
  • 437
0 votes

Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares?

Because error is being calculated according Mean-Squared-Error being minimized - as is Least-Squared technique/approach, used iteratively in Newton-method. in Python Newtons-method looks like (...
JeeyCi's user avatar
  • 210
3 votes

How Do I Calculate the Scaled Deviance of a GLM with Gamma(Exponential) Distributed Dependent Variable?

The relevant parameter of the exponential in a generalized linear model is $\mu$ (which is indeed a scale parameter); specifically we're dealing with the mean-parameterization of the exponential, of ...
Glen_b's user avatar
  • 284k
8 votes
Accepted

Advantage of GLMs over transformation models

The punchline is that the GLM is directly estimating E(y|x), whereas the transformed model is estimating E(ln(y)|x). You don't have a way of generating a prediction of E(y|x) from E(ln(y)|x): the ...
seanv507's user avatar
  • 6,952
4 votes

Advantage of GLMs over transformation models

Can you provide more details to what the book is exactly referring to? I think the important distinction between transformation models and GLMs is that for transformation models, your goal is to apply ...
Stan's user avatar
  • 385
2 votes

Why do I get a negative chi-squared value in my type III ANOVA output for my binomial GLM?

The statistic being reported is $-2\cdot\text{ln}(L_0/L_1)$, where $L$ is the likelihood of two models being compared. For type I sums of squares that would be a model without and with the parameter ...
PBulls's user avatar
  • 4,673

Top 50 recent answers are included