5
$\begingroup$

I have a multiple linear regression model with several dependent variables that have positive, negative, and zero values, and are not normally distributed. I can't do a natural log transformation because of the 0 and negative values, can't square or cube it due to 0 values, and the Box-Cox transformation works only for positive and 0 values. Is there a transformation I can do that works for all of these? I've seen log(x+minimum value) as one option, but not so much here on this forum—is this a valid transformation?

$\endgroup$
7
  • $\begingroup$ You can certainly do a log transformation by adding a constant, but why do you want to transform these data ? $\endgroup$ Feb 1, 2019 at 15:32
  • 1
    $\begingroup$ Would something like sign(x) * log(1 + abs(x)) work? It's a one-to-one transformation that has a log effect on both positive and negative values. It doesn't have issues handling x=0 either. $\endgroup$
    – jjet
    Feb 1, 2019 at 15:41
  • $\begingroup$ @RobertLong some of the variables are positively skewed. $\endgroup$ Feb 1, 2019 at 15:42
  • 3
    $\begingroup$ Squares and cubes of zero are perfectly well defined; the problem is different with squares, namely that $-x$ and $x$ produce the same square so the transformation is not one-to-one. Cubes are often a bad idea because they will typically increase (the magnitude of) skewness and exaggerate outliers. $\endgroup$
    – Nick Cox
    Feb 1, 2019 at 15:53
  • 1
    $\begingroup$ Concerning your actual problem of multiple regression, please investigate the threads on this site related to transformation of variables in multiple regression: there appear to be hundreds of them. You might begin by reviewing the highest voted and answered questions. $\endgroup$
    – whuber
    Feb 1, 2019 at 15:58

4 Answers 4

5
$\begingroup$

Yes, you can add a constant and then take a logs.

There are many ways to transform data.

There is nothing inherently invalid about doing this, but very often such transformations are misguided. It is not necessary for the dependent variable to be normally distributed. The assumption about normality concerns the residuals, not the response variable itself. If the residuals are not plausibly normally distributed then of course some transformation may be warranted.

One major downside of such transformations is that it makes sensible model interpretation much more difficult.

$\endgroup$
2
  • 1
    $\begingroup$ There is some positive skewness to some variables, which is why I wanted to transform them. But I understand that such methods may not be the best way—so is it not absolutely necessary to transform them, and to run the values as they are in the regression? $\endgroup$ Feb 1, 2019 at 15:44
  • 5
    $\begingroup$ That is not a good reason to transform them. Fit the model first, with untransformed data and do the usual regression diagnostics, including an evaluation of the residuals. $\endgroup$ Feb 1, 2019 at 15:52
4
$\begingroup$

In principle, transformations possible with variables that may be negative, zero or positive include

  • $\text{sign}(x) \log(1 + |x|)$, which conveniently preserves the sign of its argument (including mapping $0$ to $0$) while behaving like $\log x$ for $x \gg 0$ and like $-\log(- x)$ for $x \ll 0$.

  • Inverse hyperbolic sine $\text{asinh}(x)$ or more generally $\text{asinh}(kx)$ for some $k > 0$, which also preserves the sign of its argument

  • cube roots $x^{1/3}$ or more generally odd integer roots, such as also fifth roots, seventh roots, and so on, which also preserve the sign of their arguments. In practice, you may need to compute this from the product of $\text{sign}(x)$ and the chosen root of its absolute value $|x|$. Depending on your software, the root of a negative or even a zero value may be returned as missing, NA, or NaN if in practice the routine for general roots depends on first pushing the argument through code for a logarithm.

As already pointed out, whether any variable is normally distributed it is not itself important for regression, or even an ideal condition (often described as an assumption). However, pulling in outliers may be helpful for analysis of highly skewed or heavy-tailed distributions, at least for exploratory data analysis. Practical examples may include profit (negative values meaning loss) of many firms or any variable that is a change or difference.

The psychology and sociology of what a readership knows about can be important in practice in choosing one of these transformations, assuming that one such is a good idea. Thus many readers should have learned about cube roots in secondary school but inverse hyperbolic sines are more likely to seem exotic or mysterious.

$\endgroup$
3
$\begingroup$

In the case of negative values, you can use the PowerTransformer(method='yeo-johnson') method from sklearn. It is capable of handling positive and negative values, also values of zero. The skewness (measure of normality) of the data should decrease substantially. As with any transform, you should use fit and transform on your training data, then use transform only on the test and validation dataset.

pt.fit(X_train)                    ## Fit the PT on training data
X_train = pt.transform(X_train)    ## Then apply on all data
X_test = pt.transform(X_test)
X_val = pt.transform(X_val)

Example showing skewness of data decreases using PowerTransformer:

from sklearn.preprocessing import PowerTransformer

...

# find numeric features in your dataset to transform
numeric_feats = X.dtypes[X.dtypes != "object"].index

# calculate skew of all numeric features
skewed_feats = X[numeric_feats].apply(lambda x: skew(x.dropna())).sort_values(ascending=False)

# convert to dataframe for easier processing
skewness = pd.DataFrame({'Skew' :skewed_feats})

# print performance before transform
print("Pre: There are {} skewed numerical features to Box Cox transform".format(skewness.shape[0]))
print("Pre", abs(skewness.Skew).mean())

# transform data
pt = PowerTransformer(method='yeo-johnson').fit(X)
X = pd.DataFrame(pt.transform(X), index=X.index, columns=X.columns)

numeric_feats = X.dtypes[X.dtypes != "object"].index
skewed_feats = X[numeric_feats].apply(lambda x: skew(x.dropna())).sort_values(ascending=False)
skewness = pd.DataFrame({'Skew' :skewed_feats})

# print performance after transform
print("Post: There are {} skewed numerical features to Box Cox transform".format(skewness.shape[0]))
print("Post", abs(skewness.Skew).mean())

Example results:

Pre: There are 17 skewed numerical features to Box Cox transform
Pre 3.514581911418132
Post: There are 8 skewed numerical features to Box Cox transform
Post 1.977383868458546

This method was also designed to handle heteroscedastic data, which is data that has a non-uniform variance across x/y values.

Example: enter image description here

Reference: PowerTransformer from sklearn

$\endgroup$
0
$\begingroup$

I've seen log(x+minimum value) as one option, but not so much here on this forum—is this a valid transformation?

Yes and no. It depends on what you mean by "minimum value." Is there actually a minimum value for your variable? If so, this is valid. As an example, if you have counts as your data, e.g. "number of customers" or something else that's always a natural number, thenlog(1 + x) is a perfectly valid transformation for your predictor that prevents the log(0) problem. Or you could do something like say

But is it valid to take the sample minimum when there is no minimum value in the population? Emphatically no. Just think about what happens when the sample size is very large--the minimum value will approach infinity, and trying to take log(infinity + x) just doesn't make any sense. (If nothing else, it makes the log transformation basically useless, because the log can be approximated by very well using a straight line when you're far from the origin; note that the curvature goes to 0 as x goes to infinity.)

In those cases, you should instead use an inverse hyperbolic sine transform, defined as x->arcsinh(x / 2 / scale) * scale, where scale can either be estimated from the data or set to be some known value. (As a rule of thumb, setting it to 1/3 of the interquartile range will usually give appropriate behavior.) This transformation behaves like the logarithm far from 0, but behaves like a straight line close to 0.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.