5
$\begingroup$

I'm working with linear mixed-effects model in Stata. Dataset has three levels of 100k observations, nested in 500 regions, nested in 70 regions.

Currently my modelling strategy is to use three-level model with random intercepts by lev2 and lev3 for (lev2 nested within lev3):

mixed y i.x1 i.x2 || lev3: || lev2:

My outcome y is log transformed to bring it more to normality (heavily skewed data on cost).

However, the dataset contains ~2% of observations where outcome is zero and cannot be log transformed.

What other solution could I use in order to be able to handle zero outcomes?

Update 1: Following this post on The Stata Blog, it might be possible to use Poisson model instead:

meglm y i.x1 i.x2 || lev3: || lev2:, family(poisson) link(log) vce(robust)

This solution however might not work in multilevel framework.

Update 2: Revisiting this problem I came across several papers that used negative binomial regression to analyze cost. Example can be found here (with more on PubMed

Langton JM, Reeve R, Srasuebkul P, Haas M, Viney R, Currow D, et al. (2016) Health service use and costs in the last 6 months of life in elderly decedents with a history of cancer: a comprehensive analysis from a health payer perspective. Br J Cancer, 114(11):1293–302. http://dx.doi.org/10.1038/bjc.2016.75

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Are the zeros real zeros? Or do they indicate that the cost is not known or was very small (say, below 100 or below 1 or something like that)? There are of course other transformations such as square-root transformation (or more generally Box-Cox), which actually is going to end up doing something similar to assuming a Poisson model. $\endgroup$
    – Björn
    Commented Apr 26, 2016 at 13:19
  • $\begingroup$ Thanks Björn. Zero is legitimate zero cost - it's plausible to have it and looking at other variables it does make sense as in which categories these zeros appear. $\endgroup$
    – user22
    Commented Apr 26, 2016 at 15:06
  • 1
    $\begingroup$ In that case the idea of Poisson (or probably more appropriately Negative Binomial regression) is sensible, so are transformations that do not have a problem with zeros (whether it's log(x+1) or sqrt(x) or something else like 2*sqrt(x+3/8)). $\endgroup$
    – Björn
    Commented Apr 26, 2016 at 16:36
  • $\begingroup$ Poisson was indeed an option. I still have two problems with it however. Firstly, at the Stata blog post I linked to there is no mention if such approach would be suitable for multilevel framework. Secondly - when I tried anyway - the models fail to estimate. $\endgroup$
    – user22
    Commented Apr 27, 2016 at 7:02
  • 1
    $\begingroup$ I don't know whether Stata can do it (I'd assume so, but if not e.g. SAS PROC GLIMMIX or PROC NLMIXED will do it), but in principle it is possible unless there is specific issues in the data set that make the model hard to estimate. One adds a random effect (e.g. a normal one) in the $\log \mu = x\beta$ and in principle this should work, assuming there is sufficient data (same for negative binomial). Good starting values for parameter estimates may be important or specific options (e.g. number of quadrature points). $\endgroup$
    – Björn
    Commented Apr 27, 2016 at 7:23

2 Answers 2

Reset to default
7
+50
$\begingroup$

This is a great use case for the inverse hyperbolic sine transformation [1], [2]: $$ \log\left( y + \sqrt{y^2 + 1} \right) $$

Except for very small values of y, the inverse sine is approximately equal to log(2yi) or log(2)+log(yi), and so it can be interpreted in exactly the same way as a standard logarithmic dependent variable. ... But unlike a log variable, the inverse hyperbolic sine is defined at zero.

  1. http://chrisblattman.com/2011/11/15/if-you-know-what-ln1income-is-and-why-its-a-headache-you-should-read-this-post
  2. http://worthwhile.typepad.com/worthwhile_canadian_initi/2011/07/a-rant-on-inverse-hyperbolic-sine-transformations.html
Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ Wow. Very neat indeed! Never heard of that solution. And this is one more reason why I love SE! Thanks for sharing. $\endgroup$
    – user22
    Commented Apr 30, 2016 at 15:48
  • 2
    $\begingroup$ Thanks. I did not know about this transformation. It is better than my solution. As one of the links hints at, whether you want to use it is likely to depend on your expectation about your audience's sophistication: in some cases, the advantage of good behaviour at 0 of the inverse sine transformation might not outweight the fact that it will be unfamiliar to your audience. $\endgroup$
    – Antoine Vernet
    Commented Apr 30, 2016 at 16:13
4
$\begingroup$

It is not uncommon to offset a variable by a constant prior to taking its log when it cannot be log-transformed directly (for example because some observations are 0 and would map to $-\infty$).

For example, you could offset all of the observations for the variable by 1 and therefore have a log-transformed variable that is a positive reel. All the observation that are 0 in your original dataset will become $log(0 + 1) = 0$ You can find a thorough description of this issue in this related question.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks Antoine, particularly for link to other question - missed it. $\endgroup$
    – user22
    Commented Apr 26, 2016 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.