Multiple Linear Regression coefficents

Question

I'm doing a linear regression, in R. The values are like this -

u  <-  c(1,2,3,4,5,6,7,8,9,10)
v <- c(21,22,23,24,25,26,27,28,29,30)
w <- c(41,42,43,44,45,46,47,48,49,50)
y <- c(128.2305,132.4040,140.1732,147.3236, 154.5410, 158.7206, 165.1761, 169.7121,178.9751,181.0309)

If I call linear regression function, it's returning a model, which is disregarding v and w.

model <- lm(y~u+v+w)

Coefficients:
(Intercept)            u            v            w   
    122.074        6.101           NA           NA  

summary(model)

Output:

Call:
lm(formula = y ~ u + v + w)

Residuals:
       Min       1Q   Median       3Q      Max 
-2.05143 -0.92734  0.04845  0.73362  1.99357 

Coefficients: (2 not defined because of singularities)
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 122.0743     1.0197  119.72 2.65e-14 ***
u             6.1008     0.1643   37.12 3.04e-10 ***
v                 NA         NA      NA       NA    
w                 NA         NA      NA       NA    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.493 on 8 degrees of freedom
Multiple R-squared:  0.9942,    Adjusted R-squared:  0.9935 
F-statistic:  1378 on 1 and 8 DF,  p-value: 3.04e-10

I tried to fit a linear model before with different values of y,u,v (with two predictor variables, w was absent), and there also, v was being assigned NA, and only u was getting co-efficients. What's happening?

Answer 1

Two of your three regressors ($v$ and $w$, say) are linear combinations of the intercept column (the "1"-vector) and the third regressor ($u$, say), thus causing "perfect multicollinearity". For instance, the vector $v$ equals $u$ plus 20.

Technically speaking, the design matrix $X$ with rows $$ X_{i,.} = (1, u_i, v_i, w_i) $$ is not of full column rank, so the square matrix $(X'X)$ appearing in the least-squares solution $\hat \beta$ $$ \hat \beta =(X'X)^{-1}X'y $$ cannot be inverted.

Fortunately, R is so smart that it still provides the correct solution by dropping two of the three linearly dependent regressors. So it uses the design matrix $\tilde X$ with rows

$$ \tilde X_i = (1, u_i) $$ with the corresponding solution in your output.