I have to perform a regression. My data set is composed of 12 independent(ration) variables and 1 dependent(interval) variable. After having performed a Normality test(kolmogorov-Smirnov) it resulted the none of the variables were normally distributed. It is common practice to normalise the variables before a regression. My question is: should I normalise also the dependent(interval) variable? this represents a scale of quality. Would normalising it undermine the result or it's perfectly fine to normalise all the dataset and keep the dependent variable unnormalised?
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Regression
First of all, for linear regression your variables doesn't have to be normally distributed. What you are interested in is if the residuals are normally distributed.
Simple linear regression is defined as:
$$y_i = \beta_0 + \beta_1x_i + \epsilon_i$$
where $\beta_0$ is an intercept, $\beta_1$ is a slope and $\epsilon_i$ is a residual. The regression line is an expected value for $y_i$ observation being a linear function of the explanatory variable:
$$E(y_i) = \beta_0 + \beta_1x_i$$
and there is some variance ($\sigma$) of residuals around this value:
$$ y_i \sim Normal(\beta_0 + \beta_1x_i, \sigma)$$
The relation between your explanatory and the dependent variable has to be linear with residuals normally distributed and having a stable (i.e. homoscedastic) variance. Below you can see a diagram illustrating this (source here). As you can see, it's not $X$ or $Y$ that are normally distributed, but the residuals centred around value estimated using the linear regression model.
Normalization
The second thing is normalization. We define it as follows:
$$z_i = \frac{x_i - \mu}{\sigma}$$
so that the resulting variable $z$ has mean of 0 and standard deviation of 1. This doesn't make this variable a normally distributed one. After transformation, it has a standard scale, so if you use this kind of transformed variable you'll get a standardized regression estimates (estimates with mean 0 and standard deviation of 1, i.e. $z$-scores).
Below you can find an example of 1000 values drawn from uniform distribution (left side) and the same values "normalized" (right side). The distribution on the right side is not more "normal" than it was before, the only thing that changed is it's scale.
So the only thing that changes in here is the scale of explanatory variable. The change is helpful because it enables you to get "scale free" estimates of regression parameters, what makes their interpretation easier because the regression parameters relate to the standard deviations. The same is with normalizing the dependent variable - the only thing that changes is scale, the data stays the same. This kind of data transformations are commonly used in multilevel modelling where centering (i.e. subtracting the mean) is commonly used.
Check those online lectures to find out more on linear regression.
