0
$\begingroup$

I've got this model:

model <- lm (time~radius_mean+texture_mean+perimeter_mean+area_mean
             +smoothness_mean+compactness_mean+concavity_mean
             +concave_points_mean+symmetry_mean+fractal_dimension_mean+radius_se
             +texture_se+perimeter_se+area_se+smoothness_se+compactness_se
             +concavity_se+concave_points_se+symmetry_se+fractal_dimension_se
             +radius_worst+texture_worst+perimeter_worst+area_worst+smoothness_worst
             +smoothness_worst+compactness_worst+concavity_worst+concave_points_worst
            +symmetry_worst+fractal_dimension_worst+tumor_size+lymph_node, model)
summary(model)

And I want to check what transformations should I do over the variables. I tried a Box-Cox to check if a transformation over the response variable would be necessary:

require(MASS)
boxcox(model, plotit=T)
boxcox(model, plotit=T, lambda=seq(0.2,0.7,by=0.05))

But the graph says no. At this point, how can I check if a transformation over the independent variables is necessary?


Thank you for your answer. Maybe I explained myself wrong. I just want to consider if it is necessary to make any transformation over a variable and what alternative models would should be applied.

The point is that I am fully lost about transformation and I don't know how to check it with this huge amount of variables.


Thank you! I've done what you said. Here is a graph: enter image description here

I guess I should make a response transformation. So what would be the next point for this?

$\endgroup$
  • $\begingroup$ What transformations are available, how to execute them, and which to choose from is a much broader topic. It is also difficult to provide guidance without knowing what type of data you have, what the distribution of the response looks like, and what you want to do with the results (ie do you need to transform results back to original). You should also have a closer look at @Nick Cox's answer bellow, there are some troubling things about your model. After that, choose an answer to this question and repost a separate transformation question $\endgroup$ – JustGettinStarted Jan 21 '17 at 17:36
4
$\begingroup$

I am going to broaden the question by pointing out some things that worry me, some enormously, about your project.

  1. Trivially, smoothness_worst is listed twice as a predictor.

  2. Assuming that to be fixed, you are still throwing in 32 predictors into your model! That isn't anything except a recipe for poor statistical science. The predictors are a ragbag of size and shape measures of various kinds. Some have dimensions, but there is no dimensional thinking evident in your choice of predictors. For example, if some response, time in your case, is linear in area, it is most unlikely to be linear in perimeter too.

  3. Without knowing anything specific about your application, except a hint from some names that it might be in oncology, I am prepared to bet that you really need to thin down your predictors because they mean anything, many will be highly correlated with each other. Think in terms of groups of linear size measures, area size measures, shape measures, etc. and look carefully at their correlation.

  4. Most crucially of all, perhaps, I would guess that time is in practice always positive, in which case it's unlikely that linear regression really is the default model of choice. See e.g. this lucid post for an introduction to the argument that generalised linear models with logarithmic link are the starting point in modelling any response of this kind.

Box-Cox, its wonderful name apart, is in my view oversold. Worrying about marginal distributions should take second place in regression to choosing a functional form that makes sense for the science and statistics of your problem. That doesn't rule out transforming some of the predictors, say on dimensional grounds.

Linear regression isn't a washing machine that takes in dirty, messy data and removes the dirt and mess. You have to think your way towards a model that does justice to your data and the underlying science too.

$\endgroup$
2
$\begingroup$

Why are you investigating transforming variables at all? It's true that OLS regression makes assumptions about the distribution of the error term from the regression. Did you look at plots of the residuals and conclude they were not normally distributed?

And, if you did so conclude, why not use a method that does not make these assumptions? For instance, you could do quantile regression. It looks like you are using R where there is the quantreg package to do this.

$\endgroup$
  • $\begingroup$ I get that the classical linear model assumptions gives OLS great strength, but I would argue that the Gauss Markov assumptions are almost always enough to provide a good answer. This is especially true if the residuals exhibit autocorrelation or heteroscedasticity, if so normality (aside from symmetry) gives you exactly nothing $\endgroup$ – Repmat Jan 21 '17 at 17:17
  • 1
    $\begingroup$ The primary purpose of transforming regressors is not to achieve Normal residuals, but to create near-linear relationships with the response. $\endgroup$ – whuber Jan 21 '17 at 17:26
0
$\begingroup$

Following up Peter Flom's answer, let's make some test data.

library(data.table)
Test.Data <- data.table(cbind(x=rnorm(10000,1,2),z=rchisq(10000,df=7),w=rnorm(10000, 1, 5),
                              v=runif(10000),e=rnorm(10000)))
Test.Data[,y:=2.6*x+0.4*z+3.1*w+1.2*v+e]

Here there are several variables: x is normally distributed with mean 1 and sd 2, z has chi-squared distribution with 7 degrees of freedom, w is normally distributed with mean 1 and sd 4, v has a uniform distribution, and e will be an error term with normal distribution and a mean 0 and sd 1. Finally we compute y by assigning effect sizes to each of these variables (except e). (e.g. effect size of x is 2.6 etc.)

Now run the regression.

summary(lm(y~x+z+w+v,data=Test.Data))

The output is :

Call:
lm(formula = y ~ x + z + w + v, data = Test.Data)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.4473 -0.6731 -0.0107  0.6830  3.6788 

Coefficients:
             Estimate Std. Error  t value Pr(>|t|)    
(Intercept) -0.017225   0.027828   -0.619    0.536    
x            2.599458   0.004979  522.110   <2e-16 ***
z            0.401613   0.002663  150.822   <2e-16 ***
w            3.102698   0.002025 1532.436   <2e-16 ***
v            1.189750   0.034789   34.199   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.006 on 9995 degrees of freedom
Multiple R-squared:  0.9963,    Adjusted R-squared:  0.9963 
F-statistic: 6.701e+05 on 4 and 9995 DF,  p-value: < 2.2e-16

You can see that even though the variables, z and v are not normally distributed, their effect sizes were well estimated by the linear model. You should also take the precaution of checking the distribution of residuals from the lm model. I prefer doing this with qq-plots.

To do this, use resid which I will add to the Test.Data as r for simplicity

Test.Data[,r:=resid(lm(y~x+z+w+v,data=Test.Data))]

Check the qq-plot.

qqnorm(Test.Data[,r]);qqline(Test.Data[,r],col="red")

enter image description here

Seeing that the residuals fall almost perfectly on the the red line that means your residuals are normally distributed and you do not have to worry about it.

To summarize what I've done here and what @Peter Flom pointed out: Your variables do not have to be normally distributed (i.e. do not need to be transformed) for linear regression to give you good accurate estimates. Only your response (which you indicated is fine). You can also check your residuals to be sure this is working fine. In your case this would be :

qqnorm(resid(model));qqline(resid(model),col="red")

Let me know if this works out for you and if you need any additional information.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.