Understanding Regression vs. Means/Median Results

I am having a little difficulty understanding my results - could someone help me understand how to interpret, and if my process is sensible? Here is an example of what I am doing

I am trying to determine if drug_a, which is a synthetic hormone_a has an affect on hormone_b. First, I have log-transformed the data for hormone_a and hormone_b as they were both positively skewed. Normalization was successful.

When I compare the two groups (on/not on drug_a), the group on drug_a has a lower mean and median log10(hormone_b); this difference was significant (p<0.00) on an independent t-test assuming unequal variance (Levine p<0.05), and mann-whitney u.

At this point - I am thinking "ok, so maybe taking drug_a reduces hormone_b - cool." But, I want to take it a step further. I look to see if there are any other factors that are different between groups. As it turns out, the group on drug_a is older, and (not surprisingly) has a greater level of hormone_a.

So, to determine which factor is driving the difference in hormone_b - I performed a linear regression using hormone_b as the dependent variable. On multivariable linear regression all factors that were included (drug_a, hormone_a, and age) were predictive of hormone_b level. Great, - here is my question though:

The beta-value for hormone_a was positive which suggests that as hormone_a increases, so does hormone_b. This goes against what I expected - what am I missing here?

The big difference between the two approaches is that your t-test only looks at the bivariate relation between $drug_a$ and $hormone_b$, while your linear regression includes control variables ($hormone_a$ and $age$):

• your t-test says: on average, people using $drug_a$ have less $hormone_b$. But, as you correctly point out, this doesn't mean that there is necessarily a causal relationship from $drug_a$ to $hormone_b$, there could be other factors in play (for instance if older people on average have less $hormone_b$).

• effects in a linear regression are partial effects, keeping all other variables equal. The positive value of the $hormone_a$ coefficient means that people of the same age and same $drug_a$ status that have higher $hormone_a$ levels will (on average) have higher $hormone_b$ levels.

However, if only people who take drug_a have positive hormone_a values, a better estimate of the relationship between hormone_a and hormone_b could be obtained by taking this fact into account, eg. by fitting the following model, in which $homone_a$ only has an effect when $drug_a$ is 1:

$hormone_b = \beta_0 + \beta_1 drug_a + \beta_2 (drug_a \times hormone_a) + \beta_3 age$

And this brings us to the problem of correct model specification, which is always a problem in linear regression. For instance, how do you know the "correct" model is

$hormone_b = \beta_0 + \beta_1 drug_a + \beta_2 hormone_a + \beta_3 age$, and not eg. $hormone_b = \beta_0 + \beta_1 drug_a + \beta_2 \exp(hormone_a) + \beta_3 age + \beta_4 age^2$...

If you do not know the "true" correct specification, the results of linear regression are unlikely to give you the true effect of treatment. If what you want to estimate is the average treatment effect, there is a whole bunch of other methods you can use.

• Thank you - very clear answer. Will play around with the data and may have more questions. – user2416002 Jul 26 '14 at 19:34
• Jubo - if you could clarify something I don't exactly understand. To clarify, people taking drug_a have a greater hormone_a but not wildly different (i.e. 600 vs 400). When I perform a univariate analysis with hormone_a vs. hormone_b I am expecting that I would have a negative slope since people in the group on drug_a have a greater hormone_a and lower hormone_b. But even on a univariate linear regression I have a positive relationship between the two? Which do I trust more, the univariate linear regression association or the t-test between groups? – user2416002 Jul 26 '14 at 21:39
• From what I wrote before, I think neither tool correctly finds the effect of $drug_a$. Your situation can be due to the fact that within both groups (treated and non-treatment) people with higher hormone_a have higher hormone_b. But if you must choose between those two, you should take the regression, because it somewhat takes into account the other factors, and you should look at the coefficient of $drug_a$ to see if the drug affects the level of hormone_b. – jubo Jul 26 '14 at 22:53