The data for my variable is in the form of Z-scores only. I'd like to log transform the scores, but I don't know the mean or standard deviation in order to covert to raw scores. Can I assign an arbitrary mean and standard deviation, use that to convert to raw scores, then log transform the raw scores? Or is there some other way to log transform Z-scores? Thanks.
$\begingroup$
$\endgroup$
3
-
2$\begingroup$ Why do you want to log transform them? $\endgroup$– KarlCommented Oct 11, 2011 at 2:04
-
$\begingroup$ In answer to the question, "Why do you want to log transform the Z-scores?", I would like to use the variable as a predictor in multiple linear regression analysis (OLS). I've read that predictor variables need not be normally distributed, but I've also seen examples in which (positively) skewed predictors have been log transformed so presumably there is some advantage to this. Would you agree there is some advantage? Thank you to the people who answered my question. $\endgroup$– user6764Commented Oct 11, 2011 at 18:28
-
$\begingroup$ One typically uses a transformation such as e.g. the logarithm if it turns out that the relationship between the dependent and a predictor variable is non-linear. Have you had a look at the partial residual plot of the incriminated variable? Is non-linearity an issue in your model? $\endgroup$– user5644Commented Oct 12, 2011 at 10:41
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
A few quick points about logs
The following R code is a reminder that the log of a negative number is not a number and that the log of zero is negative infinity. Thus, if you are going to take a log of a z-score, you first need to make all values obtained greater than zero.
> values <- c(-2, -1, 0, .001, .1, 1, 10)
> data.frame(values=values, logvalues=log(values))
values logvalues
1 -2.000 NaN
2 -1.000 NaN
3 0.000 -Inf
4 0.001 -6.907755
5 0.100 -2.302585
6 1.000 0.000000
7 10.000 2.302585
Warning message:
In log(values) : NaNs produced
A simple strategy of logs on z-scores
A simple strategy for log transforming a variable is to first add a constant to the variable such that the minimum value is one. i.e., 1 + x - min(x).
The following code shows a simple example of some standardised positively skewed data. The minimum of 1 + x - min(x) is 1. Thus, the variable can be log transformed.
The plot then shows the density before and after transformation.
> set.seed(4444)
> # some skewed raw data
> x <- scale((rnorm(1000) + 3)^2)
>
> xnew <- 1 + x - min(x)
> min(xnew)
[1] 1
> min(x)
[1] -1.584252
> xnew <- log(xnew)
>
> par(mfrow=c(2,1))
> plot(density(x))
> plot(density(xnew))
But exactly what transformation should you perform?
- There is a general issue of whether a log transformation is appropriate to your data, and if so, what constant you should add to your raw data.
- Presumably if you already have z-scores, then you don't care too much about the absolute metric.
- You'll find further discussion of this issue on this question
answered Oct 11, 2011 at 6:24
-
1$\begingroup$ With only slightly more work, you can select an additive constant that makes the resulting distribution as symmetric as possible (measured in any convenient, robust way; a handy method is to match outer percentiles, such as by achieving $z_{0.90}-z_{0.50}=z_{0.50}-z_{0.10}$ after transformation). This has a much better chance of accomplishing the objective of the transformation, which frequently is to achieve symmetry. $\endgroup$– whuber ♦Commented Oct 11, 2011 at 17:26
$\begingroup$
$\endgroup$
You cannot assign arbitrary Mean and SD to covert z-score data into Raw data (x). However, you can check a shape of the distribution of z-scores by calculating skewness or kurtosis. Log-transform only useful if you're data is positively skewed. Moreover, it would be good if you explain that what is your objective? as @Karl asked. It might be helpful to visit this URL.
$\begingroup$
$\endgroup$
I understood that you wanted to log-transform your data so that it looked more "normal" (that is, more symmetric). But if that is the goal, why don't you apply a transform to your data, that makes it exactly standard normal?
Suppose you have a variable $x$, and you estimated its CDF as $\hat{F}(x)$. Then you can apply the transformation $y=\Phi^{-1}(F(x))$, where $\Phi()$ is standard normal CDF. By definition, $y$ will be standard normal.
Different algorithms, like scikit-learn quantile transformation, will do this for you.
