2
$\begingroup$

I have a table with several numerical continuous measurements and an indicator column catastrophy

id  |  num_1  |  num_2  |  ...  |  num_n  |  catastrophy

The catastrophy is actually a rate-event, that happens only 0.5% of the time.

I am trying to evaluate which of the num_i would be a good indicator ?.

I tried a version of z-test:

Let $m_i=\text{median}\{\text{num}_i|\text{catastrophy}=0\}$

$z_i=P(\text{num}_i>m_i|\text{catastrophy}=1)-\frac{1}{2}$

And then the p-value for $\text{num}_i$ is

$p_i=G(z_i)$

(Where $G(\cdot)$ is the standard Gaussian distribution function.) However, this test proved to be too rough, and yielded no significant variables ($p_i\leq0.05$)

$\endgroup$
  • 1
    $\begingroup$ Please consider using standard English spelling for 'catastrophe' in naming your variable. If it was just use of the word in text, I'd have edited to correct, but it's actually the name of one of your variables, so I thought it better to leave it to you to fix. (I find it quite distracting.) $\endgroup$ – Glen_b -Reinstate Monica Feb 7 '15 at 2:39
3
$\begingroup$

The test statistic $z_i$, calculated from the sample as a counted fraction, though it may well be a sensible measure of discrepancy with the null hypothesis, will not have a standard Gaussian distribution ($z_i$ is restricted to $±\frac{1}{2}$, whereas around 60% of standard Gaussian variates fall outside that range). Continuing your approach seems likely to end up with Mood's Median Test, or something similar. However, predicting the probability of an event from continuous measurements is what logistic regression was invented for. Advantages include being able to use more than one of those measurements in a predictive model, not having to make assumptions about their distribution, & being able to model non-monotonic relationships with the response.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.