I am new to the linear regression. It seems very easy and interesting. I have one question (apologies if it is simple). Suppose that I need to predict blood pressure using age, weight and height. So, the multiple linear regression equation is given by:

$Y = \beta + \beta_{1} X_{1} + \beta_{2} X_{2} + \beta_{3} X_{3}$

I read a nice paper that explains the simple and multiple linear regression models (here). At the multiple linear regression section, the authors explain the relationship between the independent variables. I wonder what is the reason for that investigation? Why do we need to investigate the relationship between the independent variables? I think as I understand, we can check the relationship between the dependent and independent variables one by one in order to select the variables with a strong effect on the dependent variable! Then, we can build a multiple linear regression model based on the selected variables (Is that correct?)

Any help, please?


Generally, the relationship between independent and dependent vars matters the most and this is what the author is doing here:

Just as in univariable regression, the coefficient of determination describes the overall relationship between the independent variables Xi (weight, age, body-mass index) and the dependent variable Y (blood pressure). It corresponds to the square of the multiple correlation coefficient, which is the correlation between Y and b1 × X1 + … + bn × Xn

But it happens often that you extract some features (independent vars) where the independent vars turn out to have some dependency on each other or the effect of one independent feature is masked by the effect of another independent feature. To determine this we need to analyse the behavior or relationship among the independent vars. This has been established by the author here:

If multiple independent variables are considered in a multivariable regression, some of these may turn out to be interdependent. An independent variable that would be found to have a strong effect in a univariable regression model might not turn out to have any appreciable effect in a multivariable regression with variable selection. This will happen if this particular variable itself depends so strongly on the other independent variables that it makes no additional contribution toward explaining the dependent variable. For related reasons, when the independent variables are mutually dependent, different independent variables might end up being included in the model depending on the particular technique that is used for variable selection.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.