What does getting coefficients to unexpected direction in logistic regression signify?

I'm starting to create a model for modelling the occurrence of skin cancer in patients and the patients have many available variables that may correlate to getting skin cancer (that's what I'm studying).

First I'm building a very simple model just to test some of the variables:

fit1 <- glm(cancer ~ trt, family = binomial, data = dta)

where $trt$ is a categorical variable taking values $0$ and $1$ and it signifies whether the patient has taken beta carotene supplement ($1$) or not ($0$, plasebo medicine). One expects that beta carotene should lower the risk of getting cancer, since that's what the study is essentially about.

However, the model produces the model:

$$cancer = 0.165659 + 0.5587886 \cdot trt$$

(also these coefficients are after taking invlogit())

So this suggests that taking beta carotene increases the likelihood to get skin cancer by 55.87%. So this is opposite of what I expect.

Is this a problem with the model being too simple or what's wrong?

However, another post suggests that I may have been computing the invlogit() of the coefficients wrongly. What I should perhaps do is calculate

$$-1.6167107 + 0.2362472 * 1$$

in logit-domain and then invlogit this sum

> invlogit(-1.380464)
 0.2009345

so this says that if taking beta carotene then a patient has 20% risk of getting skin cancer. And

> invlogit(-1.6167107)
 0.165659

i.e. 16% chance if not taking beta carotene.

Are these more reasonable?

As suggested in the comments, I've plotted:

• Can you plot the distribution of beta carotene intake for those with cancer and without and compare the distributions? I imagine you'll see essentially what the model is telling you but it doesn't hurt look, and at least this would encourage you to focus on the data rather than the model. Could it be that those who have skin cancer are taking more beta carotene as a treatment? Oct 19 '16 at 13:20
• This doesn't mean a 56% increase. Rather, the log-odds increases by 0.56. Oct 19 '16 at 14:35
• Also, if both values are categorical, logistic regression could be overkill. Try running table(dta$cancer,dta$trt) which will be more informative than the plots you have shown (which are not actually distributions). Oct 19 '16 at 14:39
• @jwimberley How do I plot their distributions then? Oct 19 '16 at 14:39
• hist(dta$cancer) and hist(dta$trt) will produce the distributions. Oct 19 '16 at 14:40

You are comparing 20% risk under treatment and 16% risk under no treatment. That's plausible. The calculation is possibly a minor issue.

Note that the canonical parameter from a logistic model is the odds ratio. You just exponentiate (not inverse logit transform) the coefficient to get the odds ratio. That is

$$exp(0.2362472) \approx 1.3$$ or that treatment is associated with 1.3 times the odds under no treatment. You can confirm this with your probabilities

$$odds(trt)/odds(no~ trt) = (0.2/(1-0.2))/(.16/(1-.16)) \approx 1.3$$

The larger issue is that you don't indicate whether or not this is a clinical trial of the effect of treatment. In the case that patients choose their own treatment, there may be factors that are associated with that choice that are also associated with the outcome. For example, if really pale individuals were more likely to choose treatment, you might observe an odds ratio > 1 even if treatment had no effect (or a beneficial effect) on skin cancer. This is referred to as confounding.

Another issue is sample size. In a small population, you are more likely to see extreme or unexpected results simply because it is easy for one or two unusual individuals to influence the results (more exactly, you are likely to have a non-representative sample). This influences statistical precision. If you have an odds ratio of 1.3 with confidence intervals that span from 0.1 to 10, then you may be just seeing statistical 'noise.'

• Does looking at the odds ratio correspond of just comparing $0.20/0.16 = 1.25 ≈ 1.3$? I.e. that from "base level" (0.16, no treatment) it's 30% more likely to get cancer with treatment. Oct 19 '16 at 17:41
• That is the risk ratio: $Pr(Y|trt)/Pr(Y|no trt)$ The risk ratio and the odds ratio will be approximately equal when the risk is low. You should be able to get answers closer to each other if you calculate the probabilities to further decimal places. Oct 19 '16 at 17:43
• I personally think the risk difference 0.2-0.16 =0.04 is the most interesting effect measure, but this is more about personal preference than anything else. Oct 19 '16 at 17:44
• However, I still don't understand why I'm seeing treatment creating more risk for cancer. I mean, what's the reason for this? Too less variables, noise in data, ...? Oct 19 '16 at 17:44
• That's not something I, or anyone, can answer definitively. You may have confounding. I would start by looking at other variables associated with both the outcome and the treatment, and I would also get confidence intervals around the odds ratios. Oct 19 '16 at 17:45