# How to correctly transform variables in linear models using residual plots?

So I have a linear model as follows;

 Mod=lm(PRICPTM ~ DISTANCE + PCTLOAD + ORIGIN + MARKET + DEREG + PRODUCT,data=x)
summary(Mod)
residualPlots(model = Mod)


The residual plots didn't look linear and about the mean. SO I transformed the dependent variable by trial and error to the following.

 Mod=lm(log(PRICPTM) ~ DISTANCE + PCTLOAD + ORIGIN + MARKET + DEREG + PRODUCT,data=x)
residualPlot(model = Mod) So now this one looks good. But the partial residual plots are still off.  What I need to do is;

1) transform the independent variables to correct the model.

2) Understand if there is any standard technique available for transforming the variables by looking at the residual plots (ie, if residual plot is sloping upwards, do this, if it is exponentially increasing, do this etc...)

• Whats wrong with partial plots? Nice linear relationships! Where is some nonlinear relationship between log(PRICPTM) and DISTANCE but linear approximation is still reasonable. If you are not satisfied with this linear approximation you can use gam model by the following code library(gam) gam.model <- gam(log(PRICPTM)~s(DISTANCE) + ORIGIN + MARKET + DEREG + PRODUCT,data = x) plot.Gam(gam.model,se=TRUE) – Dato Gogolashvili Dec 1 '19 at 15:11

Your residual plot for the log-linear model looks fairly reasonable to me, and although there are some minor issues with the partial residuals in the added-variable plots, these are an artifact of two underlying issue in your model. Firstly, if you have a look at your data frame, you will see that the variable PRODUCT takes only three values, and they do not appear to have an effect that is proportionate to its value; in effect, this is a categorical variable. By treating this as a continuous variable you impose an unnecessary constraint on the effects of these categories. Secondly, even after this change, the DISTANCE variable does indeed appear to have a nonlinear relationship with the price point. Taking a second-degree polynomial (quadratic function) gives a reasonable approximation to this relationship that improves the fit of the model and improves the residual and added-variable plots.

#Load libraries and set theme
library(MASS);
library(car);
library(ggplot2);
THEME <- theme(plot.title    = element_text(hjust = 0.5, size = 14, face = 'bold'),
plot.subtitle = element_text(hjust = 0.5, face = 'bold'));

DATA$$PRODUCT <- factor(DATA$$PRODUCT);

#Fit the log-linear regression model
FORMULA <- formula(log(PRICPTM) ~ poly(DISTANCE, 2) + PCTLOAD + ORIGIN + MARKET + DEREG + PRODUCT);
MODEL   <- lm(FORMULA, data = DATA);

#Generate predicted values and studentised residuals
DATA$$Pred <- predict(MODEL); DATA$$Res  <- studres(MODEL);

#Generate residual plot
ggplot(aes(x = Pred, y = Res), data = DATA) +
geom_point(colour = 'blue') +
geom_hline(yintercept = 0, colour = 'red', linetype = 'dashed') + THEME +
ggtitle('Residual Plot') +
labs(subtitle = '(Log-linear model for Price Point)') +
xlab('Predicted log-Price-Point') + ylab('Studentised Residuals');

#Generate residual density plot
ggplot(data = DATA, aes(x = Res)) +
geom_density(fill = 'blue') + expand_limits(x = c(-4, 4)) +
stat_function(size = 1.2, fun = dt, colour = 'red', linetype = 'dashed',
args = list(ncp = 0, df = nrow(DATA)-1)) + THEME +
ggtitle('Residual Kernel Density Plot') +
labs(subtitle = '(Log-linear model for Price Point)') +
xlab('Studentised Residual') + ylab('Density');  We can see that the residual plot displays no evidence of nonlinearity or heteroscedasticity. The residual density plot shows some evidence of non-normality, but it is not a severe departure. We can generate added-variable plots for this model to confirm that the explanatory variables have the appropriate functional relationship with the response variable. For our DISTANCE variable we have used a second-degree polynomial fit, so we have two model terms corresponding to the linear part and quadratic part of this relationship. The added-variable plots for these parts are shown below.

#Generate added variable plots
AVDATA <- avPlots(MODEL);

DATA.1 <- data.frame(AVDATA[]);
names(DATA.1) <- c('DISTANCE1','LOGPRICPTM');
ggplot(aes(x = DISTANCE1, y = LOGPRICPTM), data = DATA.1) +
geom_point(colour = 'blue') +
geom_smooth(method = 'lm', se = FALSE,
color = 'red', formula = y ~ x, linetype = 'dashed') + THEME +
labs(subtitle = '(Log-linear model for Price Point)') +
xlab('DISTANCE (linear part)') + ylab('Partial Residuals');

DATA.2 <- data.frame(AVDATA[]);
names(DATA.2) <- c('DISTANCE2','LOGPRICPTM');
ggplot(aes(x = DISTANCE2, y = LOGPRICPTM), data = DATA.2) +
geom_point(colour = 'blue') +
geom_smooth(method = 'lm', se = FALSE,
color = 'red', formula = y ~ x, linetype = 'dashed') + THEME +
labs(subtitle = '(Log-linear model for Price Point)') +
xlab('DISTANCE (quadratic part)') + ylab('Partial Residuals');  The added-variable plots show that the use of a second-degree polynomial for the DISTANCE variable is sufficient to attain linearity in the relationship between each model term and the response variable. It is possible to use a more complicated model form, but the present model appears to be sufficient to give a reasonable fit, with a reasonable form for the relationship between the distance and the price point.

Assuming you are happy with this model form, the summary output of the model is shown below. Under this model, the explanatory variables explain $$R^2 = 91.46$$% of the variability in the log-price point. This means that you should be able to make quite good predictions of the price point based on observed values of the explanatory variables. (However, to test this, you would need to do a more detailed analysis involving fitting the model to training data and then making predictions on test data.)

#Generate the model output
summary(MODEL);

Call:
lm(formula = FORMULA, data = DATA)

Residuals:
Min       1Q   Median       3Q      Max
-0.57447 -0.17764  0.00088  0.14463  0.68497

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)        11.2907630  0.0597335 189.019  < 2e-16 ***
poly(DISTANCE, 2)1 -5.2274966  0.2582329 -20.243  < 2e-16 ***
poly(DISTANCE, 2)2  2.9602807  0.3108032   9.525  < 2e-16 ***
PCTLOAD            -0.0098433  0.0005868 -16.776  < 2e-16 ***
ORIGINMIA          -0.0014046  0.0544910  -0.026 0.979476
MARKETSMALL        -0.0557069  0.0450712  -1.236 0.218785
DEREGYES           -0.9792830  0.0434109 -22.558  < 2e-16 ***
PRODUCT150          0.2019289  0.0552353   3.656 0.000376 ***
PRODUCT200          0.3153911  0.0519552   6.070 1.41e-08 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.2478 on 125 degrees of freedom
Multiple R-squared:  0.9148,    Adjusted R-squared:  0.9093
F-statistic: 167.7 on 8 and 125 DF,  p-value: < 2.2e-16

• Hi, Thanks. I got time to go through this just now. One question. Just by looking at the residual plot, can we know what transformation is to be made? Like how you gave quadratic form to DISTANCE variable in this problem, is there a standard set of things that we can try out for attaining the perfect model? or is it just trial and error? – Dom Jo Dec 26 '19 at 9:46
• And one more thing. Can you please explain how to make inferences from Added Variable plots? – Dom Jo Dec 26 '19 at 10:04
• @DomJo: If you fit a model with just the linear term, and then generate an added-variable plot, you should see that the residuals are curved, and the trend is given roughly by a quadratic. – Reinstate Monica Dec 26 '19 at 10:27
• There is another function called component + residual plot in car package. What does it give compared to avplot? – Dom Jo Dec 26 '19 at 10:51
• Never heard of it. – Reinstate Monica Dec 26 '19 at 22:32