Return to Answer

added 329 characters in body

Source Link

edited Dec 21, 2017 at 18:27

37.3k
25
148
244

@gung had a great answer. I just want to add more explanations on why "no matter what threshold you use, you will have perfect accuracy""no matter what threshold you use, you will have perfect accuracy"

In the output there are 100 probability predictions on 100 data points. The first 50 data point's probability is 0s and the second half are 1s. Among all these 100 numbers, there are 2 uniqe values 0 and 1. If we select any threshold between 0 and 1, we will always have the perfect cut, exact same label as ground truth.

Source Link

created Dec 21, 2017 at 18:21

Haitao Du

37.3k
25
148
244

@gung had a great answer. I just want to add more explanations on why "no matter what threshold you use, you will have perfect accuracy"

If we add one more line to @gung's code to check the predicted probability, we can see this: essentially for all the data points the predicted probability ether 0 or 1, this is why the threshold does not matter and we got 1 on AUC.

> predict(m,data.frame(x=x), type="response")
           1            2            3            4            5            6            7            8            9           10           11           12           13           14 
2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 4.384945e-10 2.220446e-16 5.245702e-12 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 
          15           16           17           18           19           20           21           22           23           24           25           26           27           28 
2.220446e-16 2.220446e-16 2.220446e-16 4.719935e-11 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 2.669394e-13 2.220446e-16 1.365883e-10 2.220446e-16 6.992038e-13 
          29           30           31           32           33           34           35           36           37           38           39           40           41           42 
2.220446e-16 5.435395e-12 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 5.922012e-12 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 2.220446e-16 
          43           44           45           46           47           48           49           50           51           52           53           54           55           56 
2.220446e-16 2.220446e-16 2.912038e-12 2.220446e-16 2.220446e-16 1.258165e-11 2.220446e-16 2.220446e-16 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
          57           58           59           60           61           62           63           64           65           66           67           68           69           70 
1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
          71           72           73           74           75           76           77           78           79           80           81           82           83           84 
1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
          85           86           87           88           89           90           91           92           93           94           95           96           97           98 
1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 1.000000e+00 
          99          100 
1.000000e+00 1.000000e+00