3
$\begingroup$

I'm running a multiple linear regression. Let's suppose I really need to use the logarithmic transformation. However, all values of one variable are negative. I assume I have to do the following: X1 + constant. After that I can use the logarithmic transformation and run a multiple regression.

I'd like to mention that I have done that before without the logarithmic transformation, running a simple linear regression and it has affected only alpha coefficient (makes perfect sense for me).

For example, I have got the following results:

  • y = 1,08 + 0,56*x1, original x1
  • y = -0,03 + 0,56*(x1 + 2), x1 + constant

So I can use both equations for prediction, getting the same results.

Is it still possible to interpret Beta coefficients? I am used to relying on elasticity and logarithmic transformation, showing how variables influence "Y". Do I need to take into account that I have added a "constant"? If I do, how?

Improve this question
$\endgroup$
1
  • $\begingroup$ I apologize for not being word-perfect in English. I hope I will be able to explain my future issues without any ambiguity. $\endgroup$ – Suprema tism Jan 6 '20 at 17:13
2
$\begingroup$

I would not do this. The problem is that what you choose to add to make x positive is arbitrary and can have a huge effect on the parameter estimates.

First, let's set up x and y and the model:

set.seed(1234)  #Sets a seed

x <- rnorm(100, -10, 1) #Normal mean = -10, sd = 1
y <- 3*x + rnorm(100)

Now, we'll adjust x to be positive so that logs can be taken. Usually, people choose to make the smallest adjusted x close to 0, but how close? Let's try two variations:

xadj1 <- x-min(x) + 0.01
xadj2 <- x-min(x) + 0.1

Now, we fit models:

m1 <- lm(y~log(xadj1))
summary(m1)  #-32.52 + 3.29*log(xadj1)

m2 <- lm(y~log(xadj2))
summary(m2)  #-33.90 + 4.88*log(xadj2)

And the results are quite different.

Improve this answer
$\endgroup$
14
  • 1
    $\begingroup$ @ Peter Flom - Reinstate Monica, I appreciate your help. You clarified one of the aspects of my question very thoroughly. $\endgroup$ – Suprema tism Jan 6 '20 at 11:29
  • 3
    $\begingroup$ While I agree with this, I would add for the OP that if all values of $x$ are negative then $\log |x|$ is well-defined and not arbitrary like $\log(x + c)$. . Whether this also makes statistical or scientific or practical sense for your problem cannot be said. Nor is it clear why you "really need" logarithmic transformation for such a variable. What is the specific reason? $\endgroup$ – Nick Cox Jan 6 '20 at 12:14
  • 1
    $\begingroup$ Sure, but such goals don't automatically mean that all variables must be transformed, or transformed in the same way. Context would help here and is sorely missing. That is, a transformation should ideally make physical (biological, economic, whatever) sense too. A mention of elasticity raises the odds that you are in economics. If your anonymous variable could be ever be zero or positive then $\log |x|$ is wrong in principle as well as in practice. There are alternatives such as cube root, asinh and neglog for variables that are, or could be, negative, zero and positive. $\endgroup$ – Nick Cox Jan 6 '20 at 13:22
  • 1
    $\begingroup$ We are just back where you started: you prefer logarithms. But it's still unclear why you think logarithms, or indeed the idea of elasticity, apply to data that are negative. $\endgroup$ – Nick Cox Jan 6 '20 at 15:36
  • 1
    $\begingroup$ Cube is not in my experience ever going to be an alternative to logarithm. Cube root is what was mentioned, Also, nowhere did I advise or imply that you must apply the same transformation to response and predictor. You can't move away from logarithms and keep the idea of elasticity in its simplest sense. Or more positively, elasticity will be a variable, not a constant for any curve you fit using a non-logarithmic transformation. $\endgroup$ – Nick Cox Jan 6 '20 at 16:49
2
$\begingroup$

In regression equation

$$ y = \alpha + \beta x + \varepsilon $$

the $\beta$ parameter is about the slope of the regression line, while $\alpha$ is about moving it vertically along $y$-axis. Since a picture is worth a thousand words, you can see this on a picture below showing $x$, and shifted variants of it, on $x$-axis, and $0.56 x$ on $y$-axis. As you can see on the image, in each case the slope is the same, just the lines are shifted. So the slope, and it's interpretation, remain the same.

enter image description here

As you noticed, in your example both regression formulas give same results, because they are the same:

$$\begin{align} y &= -0.03 + 0.56 (2 + x) \\&= -0.03 + 0.56 \times 2 + 0.56 x \\&= -0.03 + 1.12 + 0.56 x \\&\approx 1.08 + 0.56 x \end{align}$$

where this is approximate only because of rounding error, on unrounded values it's exact, so you get same results. The slope would take care of all such constants, also in multiple regression.

Honestly, I don't follow what you mean by the part where you mention logarithms, but if you transform $x+c$ with some function, like logarithm, the influence on the outcome gets more complicated. However adding a constant should not make any drastic differences in terms of interpretability. If in doubt, try plotting such functions against different values to check what happens (see two examples below).

enter image description here

Since it may be unclear, not every function is additive, so if you add a constant and transform, this does not mean that you could always copy and paste the slope to the equation with different transformation and get the same result. If by "same interpretation" you mean "same slope", this would not be the case for every transformation.

Improve this answer
$\endgroup$
15
  • $\begingroup$ Thanks for your answer! The part where I mention logarithms: If I have the following equation: ln(y) = a + B1*ln(x1) + B2*ln(x2+2), can I say that the coefficient B2 = elasticity of X2 $\endgroup$ – Suprema tism Jan 6 '20 at 0:01
  • 1
    $\begingroup$ @Suprematism, as said before, the slope still remains the slope. If you want to convince yourself, try plotting a number of such transformations (like logarithm, square, 1/x etc) against $x$ shifted by different constants and see what happens. $\endgroup$ – Tim♦ Jan 6 '20 at 0:13
  • $\begingroup$ Thanks again. Sorry for asking you twice. I should have understood it on the first try. I`ve just wanted to make sure. $\endgroup$ – Suprema tism Jan 6 '20 at 0:19
  • $\begingroup$ @Tim, I'm not sure thats correct. Actually I'm pretty sure the way I interpret what Suprematism is saying the slope will change. $\endgroup$ – Jesper for President Jan 6 '20 at 0:25
  • $\begingroup$ @ Stop Closing Questions Fast, would you mind explaining why? $\endgroup$ – Suprema tism Jan 6 '20 at 0:41
0
$\begingroup$

My recommendation is to apply a Box-Cox analysis of transformation using the two-parameter option (as the modeling will suggest the proper power transform and additive constant). Here is an easy discussion source and also at here.

More advanced is this paper and also the discussion in Wikipedia, especially the first example noting plots (b) and (c) which illustrates the selection process for the best power transform and additive constant based on the log-likelihood.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Some implementations fall over or baulk at negative values. More generally, despite its wonderful name (Sir David Cox is not a relative) Box-Cox seems to me oversold as a way of selecting a transformation. Almost always there are about one or two serious candidates and it's a matter of trying them and thinking about interpretation, what the audience will buy, and so forth. $\endgroup$ – Nick Cox Jan 6 '20 at 13:00
  • 1
    $\begingroup$ This does not seem to answer the question. $\endgroup$ – Tim♦ Jan 6 '20 at 14:49
  • $\begingroup$ Added a good advanced paper that covers some of the short comings of the original Box-Cox transform with modifications. My experience with starting with some known transformations of random deviates (produced by a Monte Carlo CDF inversion method) displays good results, but true it does not solve all the ills of some real world data. $\endgroup$ – AJKOER Jan 6 '20 at 16:02
  • $\begingroup$ Tim: The Box-Cox or variations thereof, is a more formal and accurate method of dealing with data that just assuming the validity of a log transform and selecting an additive constant. $\endgroup$ – AJKOER Jan 6 '20 at 16:05
  • 1
    $\begingroup$ @AJKOER but the question does not ask about alternatives. Moreover, in many cases there are transformations that make interpretability easier, then ones like Box-Cox. There's no reason for Box-Cox to be "more accurate". $\endgroup$ – Tim♦ Jan 6 '20 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.