3
$\begingroup$

I was reading the book Introduction to Statistical learning, where in z statistic was used to for hypothesis testing for parameters, so my doubt is that as response variable is not normally distributed here (at all) so parameters basically get their distribution from the response variable assumption of normality, right? so how come Z statistic exists,for parameter that is not normal?

$\endgroup$
7
$\begingroup$

No. There are many situations where a statistic can be approximately normally distributed without the related variables (e.g. the outcome variable) being normally distributed. In the case of logistic regression, I think the z-statistics are based on the fact that maximum likelihood estimates are approximately normally distributed with covariance matrix equal to the inverse of the fisher information (google it). Clearly the outcome variables is not normally distributed (it's a binary variable). So, the z-statistics you see are the coefficient estimates divided by the (square root of) the correspond entry of the fisher information, which should be approximately standard normal if the true coefficient is zero.

$\endgroup$
  • $\begingroup$ Thanks for the answer, could you recommended an article or book for this part? $\endgroup$ – GeneX Feb 1 '17 at 10:47
  • 2
    $\begingroup$ Any general book on mathematical statistics should have it, e.g. Casella and Berger (or even an undergraduate level book). Look at the wikipedia section for a basic idea en.wikipedia.org/wiki/… $\endgroup$ – gammer Feb 2 '17 at 4:08
4
$\begingroup$

Well, you can define the $Z$ statistic by the same formulas (well, actually using asymptotic approximations) that is used in the normal case. There is no implication that the resulting statistic do have a normal distribution. It might have so, but only as an asymptotic approximation.

And, in the logistic regression case, that approximation is often very bad, you can try some simulations by yourself, or search this site for the Hauck-Donner phenomenon!

$\endgroup$
  • 3
    $\begingroup$ Not sure whether I've understood your explanation,a little math in the answer would be more helpful .Thankyou $\endgroup$ – GeneX Jan 29 '17 at 17:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.