10
$\begingroup$

I have following simple X and Y vectors:

> X
[1] 1.000 0.063 0.031 0.012 0.005 0.000
> Y
[1] 1.000 1.000 1.000 0.961 0.884 0.000
> 
> plot(X,Y)

enter image description here

I want to do regression using log of X. To avoid getting log(0), I try to put +1 or +0.1 or +0.00001 or +0.000000000000001 :

> summary(lm(Y~log(X)))
Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) : 
  NA/NaN/Inf in 'x'
> summary(lm(Y~log(1+X)))

Call:
lm(formula = Y ~ log(1 + X))

Residuals:
       1        2        3        4        5        6 
-0.03429  0.22189  0.23428  0.20282  0.12864 -0.75334 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)  
(Intercept)   0.7533     0.1976   3.812   0.0189 *
log(1 + X)    0.4053     0.6949   0.583   0.5910  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4273 on 4 degrees of freedom
Multiple R-squared:  0.07838,   Adjusted R-squared:  -0.152 
F-statistic: 0.3402 on 1 and 4 DF,  p-value: 0.591

> summary(lm(Y~log(0.1+X)))

Call:
lm(formula = Y ~ log(0.1 + X))

Residuals:
       1        2        3        4        5        6 
-0.08099  0.20207  0.23447  0.21870  0.15126 -0.72550 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)  
(Intercept)    1.0669     0.3941   2.707   0.0537 .
log(0.1 + X)   0.1482     0.2030   0.730   0.5058  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4182 on 4 degrees of freedom
Multiple R-squared:  0.1176,    Adjusted R-squared:  -0.103 
F-statistic: 0.5331 on 1 and 4 DF,  p-value: 0.5058

> summary(lm(Y~log(0.00001+X)))

Call:
lm(formula = Y ~ log(1e-05 + X))

Residuals:
       1        2        3        4        5        6 
-0.24072  0.02087  0.08796  0.13872  0.14445 -0.15128 

Coefficients:
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)     1.24072    0.12046  10.300 0.000501 ***
log(1e-05 + X)  0.09463    0.02087   4.534 0.010547 *  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1797 on 4 degrees of freedom
Multiple R-squared:  0.8371,    Adjusted R-squared:  0.7964 
F-statistic: 20.56 on 1 and 4 DF,  p-value: 0.01055

> 
> summary(lm(Y~log(0.000000000000001+X)))

Call:
lm(formula = Y ~ log(1e-15 + X))

Residuals:
        1         2         3         4         5         6 
-0.065506  0.019244  0.040983  0.031077 -0.019085 -0.006714 

Coefficients:
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)     1.06551    0.02202   48.38 1.09e-06 ***
log(1e-15 + X)  0.03066    0.00152   20.17 3.57e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.04392 on 4 degrees of freedom
Multiple R-squared:  0.9903,    Adjusted R-squared:  0.9878 
F-statistic: 406.9 on 1 and 4 DF,  p-value: 3.565e-05

The output is different in all cases. What is the correct value to put to avoid log(0) in regression? What is the correct method for such situations.

Edit: my main aim is to improve prediction of the regression model by adding log term, i.e.: lm(Y ~ X + log(X))

$\endgroup$
  • 4
    $\begingroup$ None of them is $\log(x)$, they're all $\log(x+c)$, so any notion of 'correctness' there is nonsense. None of them are 'correct' for $\log(x)$. To choose between them, you'd have to say more about what properties you want and what properties you're prepared to give up. What are you actually trying to achieve? $\endgroup$ – Glen_b May 19 '15 at 8:45
  • $\begingroup$ I want to improve prediction of regression model by using lm(Y~X + log(X)). For this what would be your recommendation to avoid log(0)? $\endgroup$ – rnso May 19 '15 at 9:28
  • 5
    $\begingroup$ You can't have log(X) there; you already established that. So what are you actually trying to achieve? Given you can't take log(0), what do you want to get out of the regression? Why do you want log(X) in there? What can you tolerate instead of having log(X) in there? $\endgroup$ – Glen_b May 19 '15 at 11:27
  • 3
    $\begingroup$ What is the science here? It should be a guide to what to do. $\endgroup$ – Nick Cox May 19 '15 at 11:44
  • 1
    $\begingroup$ rnso I don't see anything there that addresses the issues I raise (or more importantly, the one Nick Cox raised), nor indeed anything that would guide an answer to the question here. $\endgroup$ – Glen_b May 19 '15 at 23:21
8
$\begingroup$

The smaller the constant is that you add the larger the outlier is that you will create: enter image description here

So it is hard to justify any constant here. You might consider a transformation that has no problem with 0s, for example a third order polynomial.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Is x+x^2+x^3 equivalent to log(x)? Please see my comments in other answer for why I am trying to use log values. $\endgroup$ – rnso May 19 '15 at 7:53
  • 2
    $\begingroup$ They are not equivalent but alternatives. $\endgroup$ – Maarten Buis May 19 '15 at 8:52
10
$\begingroup$

Why do you want to plot logarithms? What's wrong with plotting the variables as they are?

One reason to work with logs is when an assumed generating distribution is log-normal, for example.

Another would be that the numbers represent scale parameters or are used multiplicatively, in which case the space in which they lie is naturally logarithmic (for the same reason that the Jeffreys prior of a scale variable is logarithmic).

Neither of these are the case. I think the right answer here is don't do it. First come up with a data-generating model, and then use your data in a way that is consistent with that.

It sounds like what you're trying to do is to add as many functions of the inputs as possible so that you get a "great fit". Why don't you add any of these functions: http://en.wikipedia.org/wiki/List_of_mathematical_functions ? Oh, you probably think many of those are ridiculous, like the Ackermann function. Why are they ridiculous? Each function of the input you add is essentially your hypothesis of a relationship. It's hard for either of us to imagine that $y$ is a function of Euler's totient function applied to $x$. This is why I'm against $y$ being a function of $\log x$. It seems equally ridiculous to me unless you explain this hypothesis to me.

Probably the only thing you're going to get by continually adding functions of the inputs is an overfitted model. If you want a model that actually validates well, you need to make good guesses and have enough data to learn a model. The more guesses you make, the more parameters you'll have, the more data you'll need.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I do not want to plot logs. I want to use variable X in regression. To get best fit, I presume we should include log and also polynomials. For that I need log values. $\endgroup$ – rnso May 19 '15 at 7:40
  • $\begingroup$ @rnso: So you're imagining that the target value is a product of these inputs? It's very strange for the target value to be related to the inputs multiplicatively when the input can be zero. $\endgroup$ – Neil G May 19 '15 at 7:42
  • $\begingroup$ Not product but sum. I am trying to use formula: lm(Y ~ X + log(X)) $\endgroup$ – rnso May 19 '15 at 7:43
  • 1
    $\begingroup$ @rnso: yes, but adding those log terms is like saying that $e^y \sim \prod x_i^{w_i}$, and why would you think that if $x_i$ can be zero? $\endgroup$ – Neil G May 19 '15 at 7:45
  • 1
    $\begingroup$ you omit the log term. You already have the coefficient of the log term: Not a Number $\endgroup$ – Caleth May 19 '15 at 8:24
3
$\begingroup$

It's hard to say with so few details about your data and only six observations, but maybe your problem lies in your Y variable (bounded between zero and one) and not in your X. Take a look at the following approach using the two-parameter log-logistic function from the drc package:

X<-c(1.000, 0.063, 0.031, 0.012, 0.005, 0.000)
Y<-c(1.000, 1.000, 1.000, 0.961, 0.884, 0.000)

library(drc)
mod1<-drm(Y ~ X, fct=LL.2())
summary(mod1)

#Model fitted: Log-logistic (ED50 as parameter) with lower limit at 0 and upper limit at 1 (2 parms)
#
#Parameter estimates:
#  
#  Estimate  Std. Error     t-value p-value
#b:(Intercept) -1.5131e+00  1.4894e-01 -1.0159e+01  0.0005
#e:(Intercept)  1.3134e-03  1.8925e-04  6.9401e+00  0.0023
#
#Residual standard error:
#  
#  0.005071738 (4 degrees of freedom)  

plot(X,Y)
lines(seq(0, 1, 0.001), predict(mod1, data.frame(X=seq(0, 1, 0.001))))

enter image description here

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

Looking at the plot of y vs x, the functional form appears to be y = 1 - exp(-alpha x), with a very high alpha. This is close to but not quite a step function and you will need a large number of polynomials to fit this data (think in terms of exp(x) = 1 + x +x^2/2! + . + x^n/n! + ...). Rearranging terms, we get exp(-alpha x) = 1-y. If you take logs now, this gives -alpha x = log(1-y). You could define a new variable z = log(1-y) and try to find the alpha that best fits the data. You still have the issue of how to handle y = 1. I do not know the context of your problem but my impression is that you would have to think about y asymptotically approaching 1 as x approaches 1 and but y never actually reaches 1.

Thinking about this some more, I wonder if the data is actually from a Weibull distribution y = 1 - exp(-alpha x^beta). Rearranging terms, we get beta log(x) = log(-log(1-y)) - log(alpha) and we can use OLS to get alpha and beta. The issue of handling y = 1 remains.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks. Good analysis. $\endgroup$ – rnso Apr 9 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.