3
$\begingroup$

Let's say I want to evaluate the predictive value of a continuous variable in the prediction of malignancy (event/status) of a tumour.

Malignant = 1 Nonmalignant = 0

In SPSS, I can run a binary logistic regression model to do so. It allows me to set a cutoff value for classification. My question is: SPSS assumes equal pretest chances and odds in both groups, and proposes a cutoff value of 0.5. However, research has shown that malignant tumours are 70 % of all tumours, and nonmalignant tumours are 30 % of all tumours. Hence, a priori, there is a chance of 70 % for a tumour to be malignant. There is, however, no way to include an a priori chance in SPSS (at least, not by my knowledge). Am I instead allowed to change the cutoff value to 0.5/(0.7/0.5)?

The rationale is that the probability for malignancy is 0.7/0.5 times larger than 0.5, and thus that, instead of changing pretest probabilities, the posttest probability could be reduced by a factor 0.7/0.5.

Is this correct, or not?

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ Check the sensitivity and specificity at the different cut-off point (between 0 to 1). ROC curve is helpful. Then find the best cut-off point. $\endgroup$
    – user158565
    Commented Oct 13, 2018 at 21:13
  • 1
    $\begingroup$ Maybe a duplicate: stats.stackexchange.com/questions/212228/… Also stats.stackexchange.com/questions/67091/… $\endgroup$
    – kjetil b halvorsen ♦
    Commented Oct 13, 2018 at 21:31
  • $\begingroup$ @a_statistician the problem with that is that that would be based on my sample, which consists of only about 30 patients. In my sample, there's about an even number of persons with a malignant tumour and persons with a benign tumour. We're not talking about a cutoff value of the variable, but in logistic regression, the cutoff point of the probability for malignancy: a tumour has a probability for malignancy ranging from 0 to 100 %, and the cutoff is somewhere in between: when there is an even amount of malignant and benign tumours in the population, that would be 50 %. But ... $\endgroup$
    – FSJ963
    Commented Oct 13, 2018 at 22:34
  • $\begingroup$ ... here the cutoff of the probability cannot be 50 %, since there are more malignant tumours in the population than benign tumours. And although that is not the case in my sample, we need to account for that fact. We need to incorporate a 70 % a priori probability for malignancy, rather than 50 %; and since this is not directly possible in SPSS, I was wondering whether I could instead change the malignancy probability cutoff: $\endgroup$
    – FSJ963
    Commented Oct 13, 2018 at 22:38
  • $\begingroup$ SPSS: "PrTP = 0.50" or "PrTO = 1.00" || PrTO * ARG = PoTO || SPSS: 1.00 * ARG = PoTO || SPSS: if 1.00 * ARG > 1.00 then MAL || 2 options to incorporate true pretest probability: (1) Increase PrTP or PrTO such that PrTO * ARG > CUTO (impossible in SPSS) (2) Decrease CUTO such that PrTO * ARG > CUTO || REAL: PrTP = 0.70 | PrTO = 2.33 || REAL: 2.33 * ARG = PoTO || REAL: if 2.33 * ARG > 1.00 then MAL || ((1) Increase PrTO: 2.33 * ARG > 1.00) || (2) Decrease CUTO: 1.00 * ARG > 0.43 NEW CUTO: 0.43 ~ P = 30 % $\endgroup$
    – FSJ963
    Commented Oct 13, 2018 at 23:14

1 Answer 1

Reset to default
5
+50
$\begingroup$

It's easy to get confused between the two different types of probabilities that you face in this type of study. One is the probability, before you've run any tests, that someone with such a tumor has the malignant type: the 70% prevalence of the malignant form for this type of tumor. It is prior, apriori probability. The second is the probability, after you've run the test to get the biomarker value, that a tumor with a particular biomarker value is malignant. That depends on the quality of your biomarker and your probability model. It is predicted probability. There may be no simple relation between these two types of probabilities.

A logistic regression model is a model of probabilities, as this answer among others on this site emphasizes. That's a model of the second type of probability: If I know the value of the biomarker, what's the probability that the tumor is malignant?

As an extreme example, say that all tumors with biomarker values < 9 were benign, all those with values > 11 were malignant, and the few tumors with values between 9 and 11 had a 50/50 chance of being malignant. So a tumor with a value of 10 has a probability of 0.5 of being malignant.

If you wanted to use a cutoff to map probabilities into yes/no predictions, then 10 could be a reasonable choice of cutoff even though it maps to the 0.5 probability cutoff in your model of that second type of probability. You would still score about 70% of tumors as malignant because about 70% of tumors have scores above the value of 10, the first type of probability. It's the prevalence of the biomarker scores among tumors, not the choice of cutoff probability itself, that's related to the overall prevalence of malignancy if you have a near-perfect biomarker like this.

That said, it is seldom wise to reduce your logistic regression probability model directly to an immediate yes/no decision about tumor type. In a clinical context there is always other information available that needs to be taken into account. And you also have to weigh the relative costs of misclassification: what are the risks of treating a benign tumor as if it were malignant, versus the risks of treating a malignant tumor as if it were benign? If you are forced for some reason to make an all-or-none decision about classification, it's those relative risks that should be informing your cutoff.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Your last but one paragraph elucidated a lot. The last paragraph is something I surely do include in my thesis, so thanks. $\endgroup$
    – FSJ963
    Commented Oct 20, 2018 at 23:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.