linear regression and logistic regression

Question

Suppose I have a variable X1, which is a score on some psychological test, that indicates whether a person will commit a crime. The variable Y is a binary outcome which indicates whether a person finally committed a crime after the test.

If I create a table by using X1 and Y, the result is shown as below.

        TEST SCORE X1                       
        0   1   2   3   4   5
Y   0   100 20  10  7   8   9
    1   6   10  4   3   2   1

The summary of logistic regression and linear regression shows X1 is statistically significant.

My question is that I wonder which of these options is the appropriate answer for this result.

1. X1 successfully indicates whether the person will commit a crime

2. With appropriate weight (slope coefficient) X1 will successfully indicate whether the person will commit a crime.

3. Relying on the test score is insufficient since the error rates are approximately 45% (20 + 10 + 7 + 8 + 9 + 6 + 4 + 3 + 2 + 1) / sum of all. However, appropriate weight (slope coefficient) X1 will successfully indicate whether the person will commit a crime.

Any comments or answers will be much appreciated.

Answer 1

(NOTE: corrected; some details in original posting were based on mistyping the data into my software.)

This may be self-study; if you so should declare it. But imagining that this were real, or realistic, what you have done appears to be to treat the test score as if it were a measurement (which would need to be discussed and defended) and then use logit (logistic) and linear regression predicting criminal or not from test score. Beyond agreeing that the results are (just!) significant at conventional levels, an important piece of evidence is a graph like this:

The red line is a linear regression result (so-called linear probability model) and the blue curve is a logit regression result. The two are close, and there is a secondary issue of which should be used, on which I almost always plump for logit. What would happen if scores were higher is purely speculative, as you have not indicated whether higher scores than 5 are possible and in any case there are no data.

But on the major question of appropriate wording, I agree with @Glen_b: any claim of "success" would be far too strong here. Again, taking the context of detecting crime literally, that context is one in which either wrong decision (convicting the innocent or letting the guilty go free) would be a very serious matter. You would need massive and repeated analyses showing that certain test scores implied very high probability of crime for this even to be worth discussing in research literature, let alone used for real. Even for high test scores the predicted probability is not even 0.3. (Looking at the details of success or not in individual cases is also important, as you are aware; your own particular success measure seems puzzling, but I won't expand on that.)

I suspect that the example is just invented, but whether intentionally or not it does underline that there can be a massive gap between statistical significance and even borderline practical plausibility.

Note. I have tinkered with your wording and hope to have respected your meaning. However, using the slope coefficient as a weight is odd wording, as there can be no other way to use regression of any flavour!

A good comment from @kjetil prompts a further look at the data. On the graph are also empirical probabilities for each distinct score as diamonds. The significant fit of both models is indeed not even matched by a consistent monotonic trend. The very different numbers at each score level have also, I think, a side-effect of making the fits appear better than they really are. So, the more you look, the less there is a trustworthy pattern here.