0
$\begingroup$

I am trying to determine the minimally important change (MIC) of a frailty instrument using an anchor-based approach outlined below.

Step 1. Fit a logistic regression model between change_in_fi (a change in the frailty measure) and srh_decline (1/0 referring to a decline in self-reported health), which is my anchor.

Step 2. Use the predicted probabilities of the above model to fit an ROC curve.

Step 3. Identify the Youden Index, i.e. point on ROC curve which maximises the sensitivity and specificity.

Step 4. I am stuck on here - how do I find out the threshold value of change_in_fi that the above Youden Index refers to? This value is my MIC.

Please find a mock dataset and code below:

library(pROC)

dat = structure(list(change_in_fi = structure(c(-0.05825, -0.0375, 
0.04575, 0.202, 0.01675, -0.1, -0.0125, -0.04775, 0.00624999999999999, 
-0.00625000000000001, -0.01875, 0.052, -0.01025, -0.025, 0.04375, 
0.00825, -0.048, 0.09575, 0.073, 0), label = "FI score", class = c("labelled", 
"numeric")), srh_decline = c(0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 
1, 1, 0, 0, 0, 0, 0, 0, 1)), row.names = c(NA, 20L), class = "data.frame")

# Step 1
mod = glm(srh_decline ~ change_in_fi, family=binomial, data=dat)

# Step 2
rocobj= roc(mod$y, mod$fitted.values)

# Step 3
coords(rocobj, "best")
```
Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Answers to this question are probably relevant: stats.stackexchange.com/q/67560/36682 Why are you fitting a regression model in this case, with a single variable? $\endgroup$
    – Calimo
    Sep 23, 2020 at 11:36
  • $\begingroup$ Thank you very much for your answer @Calimo I had intended to include other explanatory variables (age and sex) but I can see from your answer in the above link that this would result in an infinite number of possible values for my measure of interest. If no regression model is fitted, how would one go about finding the threshold value? Thanks again! $\endgroup$
    – Dani
    Sep 23, 2020 at 13:55

1 Answer 1

Reset to default
1
$\begingroup$

If you are interested in the value of your change measure, then you should build the ROC curve directly on this data, without building a model.

rocobj <- roc(dat$srh_decline, dat$change_in_fi)
coords(rocobj, "best")
#   threshold specificity sensitivity
# 1 -0.011375         0.5           1

You will notice that the ROC curve is exactly identical, as you model was only applying a linear transformation.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.