# Multiple Linear Regression coefficents

I'm doing a linear regression, in R. The values are like this -

u  <-  c(1,2,3,4,5,6,7,8,9,10)
v <- c(21,22,23,24,25,26,27,28,29,30)
w <- c(41,42,43,44,45,46,47,48,49,50)
y <- c(128.2305,132.4040,140.1732,147.3236, 154.5410, 158.7206, 165.1761, 169.7121,178.9751,181.0309)


If I call linear regression function, it's returning a model, which is disregarding v and w.

model <- lm(y~u+v+w)

Coefficients:
(Intercept)            u            v            w
122.074        6.101           NA           NA

summary(model)


Output:

Call:
lm(formula = y ~ u + v + w)

Residuals:
Min       1Q   Median       3Q      Max
-2.05143 -0.92734  0.04845  0.73362  1.99357

Coefficients: (2 not defined because of singularities)
Estimate Std. Error t value Pr(>|t|)
(Intercept) 122.0743     1.0197  119.72 2.65e-14 ***
u             6.1008     0.1643   37.12 3.04e-10 ***
v                 NA         NA      NA       NA
w                 NA         NA      NA       NA
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.493 on 8 degrees of freedom
Multiple R-squared:  0.9942,    Adjusted R-squared:  0.9935
F-statistic:  1378 on 1 and 8 DF,  p-value: 3.04e-10


I tried to fit a linear model before with different values of y,u,v (with two predictor variables, w was absent), and there also, v was being assigned NA, and only u was getting co-efficients. What's happening?

• Try summary(model) command. It may be useful to post its output here. For reproducible code in R, use proper commands like u = c(1,2,...) etc. – rnso Apr 12 '15 at 7:34

Two of your three regressors ($v$ and $w$, say) are linear combinations of the intercept column (the "1"-vector) and the third regressor ($u$, say), thus causing "perfect multicollinearity". For instance, the vector $v$ equals $u$ plus 20.

Technically speaking, the design matrix $X$ with rows $$X_{i,.} = (1, u_i, v_i, w_i)$$ is not of full column rank, so the square matrix $(X'X)$ appearing in the least-squares solution $\hat \beta$ $$\hat \beta =(X'X)^{-1}X'y$$ cannot be inverted.

Fortunately, R is so smart that it still provides the correct solution by dropping two of the three linearly dependent regressors. So it uses the design matrix $\tilde X$ with rows

$$\tilde X_i = (1, u_i)$$ with the corresponding solution in your output.

• okay. So you mean to say, the variables are like v = ku+e, or w = ku+e, so it'll be a case for multi-collinearity e is some error or intercept? – tired and bored dev Apr 12 '15 at 8:07
• no, thats not what I meant. There is no "e" in my text. $v = u + 20\cdot 1$ e.g. – Michael M Apr 12 '15 at 8:09
• Thanks so much... :) Okay. This one.. v(1) = u(1)+20; v(2) = u(2)+20; – tired and bored dev Apr 12 '15 at 8:13
• Exactly, that is right. – Michael M Apr 12 '15 at 8:43
• I liked the clarity with which you answered. Could you please suggest a good discussion on regression, with math concepts, preferably using R.. – tired and bored dev Apr 12 '15 at 9:02