2
$\begingroup$

I am trying to estimate the effect of a binary variable $B$ on a continuous outcome $y$. The outcome $y$ is non-negative and highly skewed, its max being $92$ times greater than its median.

I tried to measure the impact of $B$ on $y$ using a t-test with $4,500$ observations. The effect of $B$ on $y$ is negative with a p-value of $0.758$.

My second approach was to take the log of $y$ and run an OLS regression. I can then re-transform the coefficient for $B$. In this case it is positive with a p-value of $0.000315$.

How do I make sense of these estimates? Does $B$ increase or decrease $y$?

$\endgroup$
2
  • $\begingroup$ Can you give a rough description of how the distribution of $y$ looks when $B=1$, and when $B=0$? In particular, what ranges of values do they take? $\endgroup$
    – angryavian
    Commented Oct 3, 2020 at 22:00
  • $\begingroup$ Between 0 and 1, skewed to the right, bi-modal. $\endgroup$
    – badmax
    Commented Oct 5, 2020 at 19:50

1 Answer 1

2
$\begingroup$

Even stipulating that the causal interpretation ('increase') is valid, it's entirely possible that $B$ both increases and decreases $y$.

The original $t$-test compares arithmetic means; the log model compares geometric means. There is nothing especially weird about a binary variable increasing one and decreasing the other.

The mean is more affected by the extreme values than the geometric mean, so if $B$ decreased the extreme values a lot but increased typical values a little, you'd see the mean go down and the geometric mean go up. The geometric mean tends to behave more like the median than the mean.

For a concrete example, I have seen a dataset in which giving inhaled steroids to kids with asthma decreased the mean medical expenditure (by preventing emergency department visits and hospitalisation), but increased the median medical expenditure because of the cost of treatment. I don't know whether the geometric mean went up, but I would have expected it to.

We can also do an analytic example with log-Normal distributions. If $\log Y\sim N(\mu,\sigma^2)$ the geometric mean of $Y$ is $e^\mu$, but the (arithmetic) mean is $e^{\mu+\sigma^2/2}$. So if $B$ decreases $\sigma^2$ by an amount $x$ and increases $\mu$ by less than half $x$, the mean goes down and the geometric mean goes up.

This still leaves open the question of which answer is helpful for your question, which would require more context. In the medical-cost example it was clearly the mean that was relevant, but in some cases the geometric mean or some other summary might be relevant.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.