3
$\begingroup$

I am trying to check variable importance of this gender variable. I know if p value is less than 0.05 then its important otherwise not but what is it's giving p-value < 2.2e-16. I have tried other methods too but giving the same for all of the categorical variable. I have pasted output for only one variable. SO, how to interpret this. (variable to be considered or not).

Frequency of my data points

 table(data$gender,data$target)

 Output: 
          N       Y
  F 2107566 2560932
  M 1307442 1567399
  U       3      16

To test statistical significance score:

chisq.test(table(data$gender,data$target))

Output:

        Pearson's Chi-squared test

    data:  table(data$gender, data$target)
    X-squared = 86.9407, df = 2, p-value < 2.2e-16

Note : I think one of the possible reason might be because of 7.5 million rows. So, will this be solved by sampling the data for checking the significance.

$\endgroup$
  • 4
    $\begingroup$ This means $p< 2.2*10^{-16}$ and is effectively close to zero (actually numerically undistinguishable from 0). Do you also ask how to interpret such a small p-value? There are many questions here asking that very thing. $\endgroup$ – Momo Sep 23 '15 at 21:29
  • 2
    $\begingroup$ <2.2e-16 means $0.00000000000000022$. It is (very much) less than $0.05$. On a different note, what kind of gender is U? $\endgroup$ – gung - Reinstate Monica Sep 23 '15 at 21:31
  • $\begingroup$ Why do you think p values below 5% are important? $\endgroup$ – Michael M Sep 23 '15 at 21:36
  • $\begingroup$ @gung, U means unknown (or intersex). $\endgroup$ – Vignesh Prajapati Sep 23 '15 at 23:17
  • 4
    $\begingroup$ Relevant: stats.stackexchange.com/questions/78839/… $\endgroup$ – Glen_b Sep 24 '15 at 2:05
0
$\begingroup$

Important parts of the Q is answered. That p-value enables you to undoubtedly report on whether your hypoethsis is true or not. It is far more better than when p-value is at the critical level something like 0.05

$\endgroup$
  • $\begingroup$ hi, Still I am confused about Whether to consider this variable or not. $\endgroup$ – Vignesh Prajapati Sep 23 '15 at 23:20
  • $\begingroup$ @VigneshPrajapati Did you not understand gung's first comment? $\endgroup$ – Glen_b Sep 24 '15 at 3:53
  • $\begingroup$ @Glen_b, I did. But again I have confusion is that if <2.2e-16 is less than 0.05 then why its giving the same value for all of my categorical value. Even I check it for my index no (user_id) its giving me same value. $\endgroup$ – Vignesh Prajapati Sep 24 '15 at 12:40
  • $\begingroup$ The "<2.2e-16" isn't a value but a range of values. Because you have huge sample sizes (7.5 million?) you should not be surprised that the p-values are very low. $\endgroup$ – Glen_b Sep 24 '15 at 13:10
  • $\begingroup$ So, please answer my quick question. Should I consider those variable or not ? $\endgroup$ – Vignesh Prajapati Sep 24 '15 at 14:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.