Say I wish to do a linear model and my dependent variable $Y$ follows a binomial distribution: $Y_i \sim Bin(n_i,p_i), i = 1,...,N$. Usually I would use a generalized linear model and perform a logistic regression. But if the $n_i$'s are large enough, the $Y_i$'s will approximately follow a normal distribution, $Y_i \sim N(n_ip_i, \ n_ip_i(1-p_i))$. So in that case shouldn't I be able to simply perform a simple linear regression, without using a generalized linear model?
3
-
$\begingroup$ Here are the pros and cons to using a linear model stats.stackexchange.com/questions/304437/… $\endgroup$– Michael WebbApr 22 at 20:26
-
1$\begingroup$ Another assumption of linear regression is that the conditional responses have approximately the same variance. That's a matter of the $n_ip_i(1-p_i)$ not appreciably changing with $i.$ If that's not the case, watch out! $\endgroup$– whuber ♦Aug 23 at 19:24
-
$\begingroup$ There’s also the logistic regression link function that squeezes the predictions into the unit interval. Your linear model will not do that and could make a prediction like $-0.2$. $\endgroup$– DaveAug 23 at 20:50
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
One of the assumptions of the linear regression is homoscedasticity (constant variance). In the binomial model, the variance obviously depends on $p_i$, which is the variable you're trying to model. As long as your curve is reasonably flat, or, in other words, the $p_i$'s don't vary too much, this will probably not be an issue. But still, if I were to review your results, I'd probably ask why you hadn't used logistic regression. What are the benefits of using a model which you know is less suited than another? ("All models are wrong, but some are useful.")
-
1$\begingroup$ The idea is to use a more sophisticated model such as a linear mixed model and LASSO. Sofar the implementations in R of such advanced models for the generalized linear model did not work for me. My hope is that the implementations of such models for linear models would work. For example using the glmmLasso package I cannot add more than a few predictors, without the package running into issues. $\endgroup$ Jul 1, 2020 at 6:50
-
$\begingroup$ I see, it's a question of available software. If I were you, I'd check that, for your range of $n_i$'s and $p_i$'s, the differences between the binomial and normal distributions and the "variance of the variances" are below some pre-defined thresholds and, if they are, use the approximation. Using such approximations is not unusual in statistics: Pearson's chi-squared test, used on contingency tables, also approximates counts (integers!) with normal distribution. $\endgroup$– Igor F.Jul 1, 2020 at 7:09
-
$\begingroup$ Do you have any pointers how I could quantify if the approximation is close enough? I guess choosing an appropriate threshold would be kind of arbritrary? $\endgroup$ Jul 1, 2020 at 7:19
-
$\begingroup$ As far as I know, all thresholds are arbitrary. We use 0.05 as the significance threshold not because of some statistical theory behind it, but because we have 5 fingers on each hand. Also, statisticians say that chi-squared test can be used when the count number in each cell is at least (surprise!) 5. So I guess if you settle for 5% error, no-one will be offended. Also, people here on CV have suggested using weighted linear regression when variances are not constant (see e.g. stats.stackexchange.com/a/275929/169343 or stats.stackexchange.com/a/144238/169343). $\endgroup$– Igor F.Jul 2, 2020 at 6:46