11
$\begingroup$

I would like to know how to transform negative values to Log(), since I have heteroskedastic data. I read that it works with the formula Log(x+1) but this doesn't work with my database and I continue getting NaNs as result. E.g. I get this Warning message (I didn't put my complete database because I think with one of my negative values is enough to show an example):

> log(-1.27+1)
[1] NaN
Warning message:
In log(-1.27 + 1) : NaNs produced
> 

Thanks in advance

UPDATE:

Here is an histogram of my data. I'm working with palaeontological time series of chemical measurements, E.g the difference between variables like Ca and Zn is too big, then I need some type of data standardization, that is why I'm testing the log() function. enter image description here

This is my raw data

$\endgroup$
  • 2
    $\begingroup$ The logarithm is only defined for positive numbers, and is usually used as a statistical transformation on positive data so that a model will preserve this positiveness. The log(x+1) transformation will is only defined for x > -1, as then x + 1 is positive. It'd be good to know your reason for wanting to log transform your data. $\endgroup$ – Matthew Drury Jun 4 '15 at 4:58
  • 3
    $\begingroup$ Tell us more about the data, including the range, mean, frequencies of negative, zero and positive values. It could be that a generalized linear model with log link makes most sense for the data so long as it is reasonable to think that the mean response is positive. It could be that you should not be transforming at all. $\endgroup$ – Nick Cox Jun 4 '15 at 6:34
  • 6
    $\begingroup$ Thanks for adding details. For such data 0 has a meaning (equality!) that should be respected, indeed preserved. For that and other reasons I would use cube roots. In practice, you will need some variation on sign(x) * (abs(x))^(1/3), the details depending on software syntax. For more on cube roots see e.g. stata-journal.com/sjpdf.html?articlenum=st0223 (see esp. pp.152-3).We used cube roots to help visualization of a response variable that can be positive and negative in nature.com/nature/journal/v500/n7464/full/… $\endgroup$ – Nick Cox Jun 4 '15 at 7:54
  • 8
    $\begingroup$ Why aren't you transforming the original variables instead of the differences? $\endgroup$ – whuber Jun 4 '15 at 8:23
  • 4
    $\begingroup$ You solved the mathematical problem. @whuber's suggestion or cube roots would still, I think, be easier to work with, especially if the constant is purely empirical or varies between variables. A good rule for choice of transformations is only to use transformations that would work for similar data you can imagine. Thus $\log(x + 4)$ "works" for $x > -4$ but would fail if your next batch was bounded by $-5$.. $\endgroup$ – Nick Cox Jun 6 '15 at 7:00
13
$\begingroup$

Since logarithm is only defined for positive numbers, you can't take the logarithm of negative values. However, if you are aiming at obtaining a better distribution for your data, you can apply the following transformation.

Suppose you have skewed negative data:

x <- rlnorm(n = 1e2, meanlog = 0, sdlog = 1)
x <- x - 5
plot(density(x))

then you can apply a first transformation to make your data lie in $(-1,1)$:

z <- (x - min(x)) / (max(x) - min(x)) * 2 - 1
z <- z[-min(z)]
z <- z[-max(z)]
min(z); max(z)

and finally apply the inverse hyperbolic tangent:

t <- atanh(z)
plot(density(t))

Now, your data look approximately normally distributed. This is also called Fisher transformation.

$\endgroup$
  • 9
    $\begingroup$ You solved the immediate mathematical problem. But I don't think most likely consumers of statistical results would find it easy to think about $\text{atanh}[(x - \min(x)) / (\max(x) - \min(x))]$ as a response scale and in modelling you would need to think what error structure makes sense.The scale would be sensitive to the empirical minimum and maximum. $\endgroup$ – Nick Cox Jun 4 '15 at 6:38
  • 2
    $\begingroup$ @NickCox You are absolutely right. Maybe if the OP add more details about his problem, we could figure out an alternative solution! $\endgroup$ – stochazesthai Jun 4 '15 at 6:43
  • $\begingroup$ The inner argument in my first comment is not what is being transformed, but the spirit of my comment is I think unaffected. $\endgroup$ – Nick Cox Jun 4 '15 at 8:35
  • $\begingroup$ Dear @stochazesthai thanks for your detailed explanation, but I can't apply your code to my data. I updated my question with a link of my raw data at the end. $\endgroup$ – Darwin PC Jun 5 '15 at 6:18
  • $\begingroup$ The statements z <- z[-max(z)] and z <- z[-min(z)] inappropriately shrink z down to a single value. Also the general function atanh(((x - min(x)) / (max(x) - min(x)))) produces Inf for the minimum and maximum values of x. $\endgroup$ – Max Ghenis Aug 7 '18 at 19:39
-1
$\begingroup$

To transform it to a log scale, first find the log of the positive number then multiply it by its sign, the following code should do that.

transform_to_log_scale <- function(x){
    if(x==0){
        y <- 1
    } else {
        y <- (sign(x)) * (log(abs(x)))
    }
        y 
    }

Using the above example we can plot the following skewed distribution

x <- rlnorm(n = 1e2, meanlog = 0, sdlog = 1)
x <- x - 5
plot(density(x))

enter image description here

After using the transforming function as follows, we get a distribution that looks more 'normal'

plot(density(sapply(x,FUN=transform_logs_scale)))

enter image description here

$\endgroup$
  • 2
    $\begingroup$ (1) Most programming languages (R included) implement the signum function (which returns -1 for negative numbers, 1 for positive numbers and 0 for zero). Using it would be more expressive and faster. (2) Your proposal is a poor one for analyzing data like those illustrated, because it has a huge discontinuity at zero! $\endgroup$ – whuber Jul 29 '17 at 20:54
  • $\begingroup$ thanks for signum, i didn't know about it, wonder how it is implemented $\endgroup$ – yosemite_k Jul 29 '17 at 21:01
  • 2
    $\begingroup$ There are various ways. In many processor architectures a sign bit is set after many operations, so it could be used. In the IEEE double precision floating point representation, the sign can be found by inspecting a single bit (plus another quick test for a true zero). In pipelined architectures with predictive branching, etc., it's usually much more efficient not to branch if at all possible, which is why using the built-in version of signum can be a significant computational gain. Incidentally, setting y <- 1 when $x=0$ looks arbitrary--it could really screw up a statistical analysis. $\endgroup$ – whuber Jul 29 '17 at 21:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.