2
$\begingroup$

I have 5 variables that I've used to calculate a score with. Each variable contributes 0-2 points to the score depending on their value.

I'm interested in whether these predictors better can predict the outcome (disease status) by themselves or if a score of all would predict the outcome better.

The score works very well in a regression model and seems to be able to predict disease status with p=0.003 and a satisfactory confidence interval (CI). The 5 binary variables are also significant but with very large CI. Does this mean that the score has a higher validity than the binary variables on their own?

Also if I run a regression model with all 5 predictors, the R-squared is 0.35 and the model P value is 0.01, with only one of the predictors being significant in the model, though. Is this worth noting in the manuscript? I'm not exactly sure what it means other than that the model itself is good at predicting disease status.

DATA: https://gofile.io/?c=t8Htvb

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Does this mean that the score has a higher validity than the binary variables on their own?

Precision and validity are not the same thing. I can estimate something very precisely and have it be invalid. You'll need to determine what your hypothesis actually is before you select a regression model. Are you interested in how the composite score affects the outcome, or how each component of the score affects the outcome? The answer to this question will help inform which model you should select.

That the CIs are "very large" might also mean that the 5 predictors are highly correlated. I can't say for sure without seeing the data.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Hi Demetri I edited my question to better clarify exactly what my aim is. The CI's are large in a univariate regression for each predictor, so it's not correlation. $\endgroup$
    – Paze
    Commented Jan 16, 2020 at 16:58
  • $\begingroup$ @Paze I would avoid doing univariable regressions for model selection. I can't comment more about your scenario without your data in hand. $\endgroup$
    – Demetri Pananos
    Commented Jan 16, 2020 at 17:01
  • $\begingroup$ I've added my data in CSV if you'd like to take a look $\endgroup$
    – Paze
    Commented Jan 16, 2020 at 17:17
  • $\begingroup$ @Paze You haven't really answered my initial question: Does your hypothesis concern the predictors or the aggregate score? If all you want to do is predict disease status, then you can do whatever you want so long as you validate the model correctly. P values tell you nothing about predictability $\endgroup$
    – Demetri Pananos
    Commented Jan 16, 2020 at 17:46
  • 1
    $\begingroup$ @Paze then model the outcome as a function of the composite score, ignoring other variables. $\endgroup$
    – Demetri Pananos
    Commented Jan 16, 2020 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.