Normally distributed estimated parameters in logistic regression

Question

I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally distributed here (at all) so parameters basically get their distribution from the response variable assumption of normality, right? so how come Z statistic exists,for parameter that is not normal?

Answer 1

No. There are many situations where a statistic can be approximately normally distributed without the related variables (e.g. the outcome variable) being normally distributed. In the case of logistic regression, I think the z-statistics are based on the fact that maximum likelihood estimates are approximately normally distributed with covariance matrix equal to the inverse of the fisher information (google it).

Clearly the outcome variables is not normally distributed (it's a binary variable). So, the z-statistics you see are the coefficient estimates divided by the (square root of) the correspond entry of the fisher information, which should be approximately standard normal if the true coefficient is zero.

Answer 2

Well, you can define the $Z$ statistic by the same formulas (well, actually using asymptotic approximations) that is used in the normal case. There is no implication that the resulting statistic do have a normal distribution. It might have so, but only as an asymptotic approximation.

And, in the logistic regression case, that approximation is often very bad, you can try some simulations by yourself, or search this site for the Hauck-Donner phenomenon!