Interpretation of Coefficients in linear regression using 'fitlm'

Question

I require help with regards to the interpretation of linear regression results (I'm using the Matlab 'fitlm' function).

My data has 8 features, and when each feature is plotted against the response variable there are some obvious relationships (see figure below).

From looking at this plot I would expect features x4, x5, x6, and x7 to all have negative coefficients, and probably x8 to have a positive coefficient.

When I execute the following code:

mdl = fitlm(features,y,'linear','RobustOpts','on')

I get the following output:

                  Estimate        SE        tStat        pValue
(Intercept)         3.1936    0.038772      82.368    3.9862e-95
x1             -0.021465    0.040015    -0.53643       0.59283
x2              0.012444    0.018055     0.68919       0.49227
x3              0.014156    0.031247     0.45305       0.65148
x4               0.25286     0.09546      2.6488     0.0093614
x5                 2.378      1.2857      1.8496      0.067274
x6               -2.0413     0.25464     -8.0164     1.882e-12
x7               -1.7374      1.0649     -1.6314       0.10588
x8              -0.17522    0.031894      -5.494    2.9021e-07

which seems a bit counter-intuitive, since the coefficients of some of the features (i.e. the gradients of x4 and x5) are positive when I would expect them to be negative.

The model reports a low RMSE and a very high R-squared of 0.969, which suggests a very good fit. The pValues are quite low as well, which suggests that the features with the unusual gradients are statistically significant.

I'm wondering why this is the case and how can I interpret these results?

Answer 1

This could happen if there is correlation among the X variables. For instance, if X4 and X6 are highly correlated, then perhaps the effect of X4 on the response after adjusting for X6 is positive, even if the correlation between X4 and Y is negative.

I believe you can use something like

plotAdded(mdl,'x4')

to look at the relationship between X4 and Y after adjusting for the other predictors.