0
$\begingroup$

I am working on a research project.

We have roughly 300 study participants that have been categorised into 4 groups. Some of these participants had a recurrence of a disease, and the rest did not.

I'm trying to see if classification into any of the 4 groups has a statistically significant impact on the recurrence (y/n).

I ran a logistic regression model, using statsmodels in Python, with recurrence (yes/no) as the dependent, binary variable, and the "4 groups" as a single, independent categorical variable. It outputted coefficients, and gave me p-values, with one of the p-values being significant.

I am not sure if the model one hot encodes that categorical variable, if the model is even valid, or if you can even actually run a simple logistic regression model with 4 categories and a single binary outcome (have I actually done a multivariate logistic regression?)

Also, am I supposed to do a chi-square statistic first (comparing the categorical variable - those 4 groups - and the recurrence - yes/no), then run a simple logistic regression model only on the significant groups, each separately?

thank you so so much for reading this and for your help! I am a complete noob and I have been trying to figure this out for ages!

thank you :)

$\endgroup$
2
  • $\begingroup$ So your data is only the recurrence (y/n) & the four groups? What question are you trying to answer? $\endgroup$
    – Tim
    Sep 23, 2022 at 12:49
  • $\begingroup$ hi tim! yes that's right. I'm trying to see if classification into any of the 4 groups has a statistically significant impact on the recurrence (y/n). I should have clarified that, sorry $\endgroup$
    – user123
    Sep 23, 2022 at 12:52

2 Answers 2

2
$\begingroup$

Firstly, a key question is how the 4 groups were created. If it's treatments assigned by randomization that you want to compare, then proceed as you did (unless there's important intercurrent events) and the responses below apply. Otherwise, this might be completely inappropriate and you need to consider what (causal?) questions you are trying to answer (e.g. comparing treatments without randomization would require you to account for how the treatment assignment works) and can get very quickly very complicated (consult an experienced epidemiologist or someone e.g. a statistician with experience in causal inference from non-randomized observational studies).

You should be able to tell what happened in your model (i.e. did "one-hot-encoding" aka "dummy coding" occur?) by looking at the output. If there's a coefficient for each group (or for an intercept plus 3 of the groups, in which case the group that does not "appear" is the reference category relative to which the coefficients for the others are given), then the groups were treated as categorical. If you e.g. instead got a single coefficient for group, this would indicate "group" was treated as a continuous numerical variable (not what you want).

You need to know, ideally before the analyses are done, which groups you want to compare (i.e. what were your scientific questions?). The p-values given will be with reference to the reference group (in case you have intercept and 3 group coefficients) or comparing each group's proportion to a log-odds ratio of 0 (=proportion of 50%) in case you have coefficients for 4 groups (and not intercept). Doing other comparisons requires forming the corresponding contrasts, for which any statistics packages should have suitable functions/tools.

Comparing all the groups (or some subset of groups) results in a multiplicity problem. I.e. you do not control the familywise type I error at the whatever significance level you consider, if you simply look at the p-values given in the output. Whether that is an issue is another question, and there are many questions/discussions on this site on when and which multiplicity adjustments might be appropriate.

$\endgroup$
1
  • $\begingroup$ Hi Björn, thank you very much for your detailed answer! The participants were sorted into the 4 groups by me based on the appearance of the disease on their brain scans (at baseline), then recurrence was recorded if it occurred at some point over 30 days from baseline. Treatment was actually randomised between two different drugs, but the 4 groups had no impact on that whatsoever and was determined at a much later date. Does the result still apply? Also, I did get an intercept and 3 groups, all with a coefficient and p-values. Does that mean the p-value is in reference to the intercept? Thanks $\endgroup$
    – user123
    Sep 23, 2022 at 14:21
0
$\begingroup$

Adding to Björn's answer (+1), you have only the binary data (yes, the groups should be dummy-coded), so what you will be doing is something like comparing the average recurrence, or counts, across the groups. Notice that no matter if you just calculate average recurrence per group, use logistic regression, or even use linear regression, they will be just calculating the averages (see an example using toy data below).

> by(mtcars$am, mtcars$gear, mean)
mtcars$gear: 3
[1] 0
------------------------------------------------------------ 
mtcars$gear: 4
[1] 0.6666667
------------------------------------------------------------ 
mtcars$gear: 5
[1] 1

> by(predict(glm(am~as.factor(gear), family="binomial", data=mtcars), type="response"), mtcars$gear, mean)
mtcars$gear: 3
[1] 1.170227e-09
------------------------------------------------------------ 
mtcars$gear: 4
[1] 0.6666667
------------------------------------------------------------ 
mtcars$gear: 5
[1] 1

> lm(am~as.factor(gear),  data=mtcars)

Call:
lm(formula = am ~ as.factor(gear), data = mtcars)

Coefficients:
     (Intercept)  as.factor(gear)4  as.factor(gear)5  
      -2.355e-16         6.667e-01         1.000e+00  

To answer the question "are there any differences between groups" you can use the $\chi^2$ test, to compare the means ANOVA, etc, but what you need is just hypothesis testing.

$\endgroup$
1
  • $\begingroup$ thank you for your help! $\endgroup$
    – user123
    Sep 23, 2022 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.