Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
2 added 150 characters in body
source | link

I hope this helps :

> x1 <- rnorm(30)
> x2<- rnorm(30, mean = x1, sd=0.01)
> y <- rnorm(30, mean = 5+x1+x2)
> cor(x1,x2)
[1] 0.9999316
> fit <- lm(y~x1+x2)$coeff;fit
(Intercept)          x1          x2 
   5.175611    3.859883   -1.452410 
   > 
   > 
   > library(MASS) 
   > lm.ridge(y~x1+x2,lambda = 1)
           x1       x2 
5.171224 1.187841 1.174328 
   > 
   > 
   > 
   > 
   > x1 <- rnorm(30)
   > x2<- runif(30, min = 0, max = 1)
   > cor(x1,x2)
[1] 0.3348988
   > fit <- lm(y~x1+x2)$coeff;fit
(Intercept)          x1          x2 
  5.0691119   0.2017907  -0.6022160 
> lm.ridge(y~x1+x2,lambda = 1)
               x1         x2 
  5.0507017  0.1928616 -0.5704941  

Also, there is a theorem stated that " There always exists a value $\lambda$ such that $$MSE(\widehat{\beta_{\lambda}}) < MSE(\widehat{\beta}^{OLS})$$

I hope this helps :

> x1 <- rnorm(30)
> x2<- rnorm(30, mean = x1, sd=0.01)
> y <- rnorm(30, mean = 5+x1+x2)
> cor(x1,x2)
[1] 0.9999316
> fit <- lm(y~x1+x2)$coeff;fit
(Intercept)          x1          x2 
   5.175611    3.859883   -1.452410 
   > 
   > 
   > library(MASS) 
   > lm.ridge(y~x1+x2,lambda = 1)
           x1       x2 
5.171224 1.187841 1.174328 
   > 
   > 
   > 
   > 
   > x1 <- rnorm(30)
   > x2<- runif(30, min = 0, max = 1)
   > cor(x1,x2)
[1] 0.3348988
   > fit <- lm(y~x1+x2)$coeff;fit
(Intercept)          x1          x2 
  5.0691119   0.2017907  -0.6022160 
> lm.ridge(y~x1+x2,lambda = 1)
               x1         x2 
  5.0507017  0.1928616 -0.5704941  

I hope this helps :

> x1 <- rnorm(30)
> x2<- rnorm(30, mean = x1, sd=0.01)
> y <- rnorm(30, mean = 5+x1+x2)
> cor(x1,x2)
[1] 0.9999316
> fit <- lm(y~x1+x2)$coeff;fit
(Intercept)          x1          x2 
   5.175611    3.859883   -1.452410 
   > 
   > 
   > library(MASS) 
   > lm.ridge(y~x1+x2,lambda = 1)
           x1       x2 
5.171224 1.187841 1.174328 
   > 
   > 
   > 
   > 
   > x1 <- rnorm(30)
   > x2<- runif(30, min = 0, max = 1)
   > cor(x1,x2)
[1] 0.3348988
   > fit <- lm(y~x1+x2)$coeff;fit
(Intercept)          x1          x2 
  5.0691119   0.2017907  -0.6022160 
> lm.ridge(y~x1+x2,lambda = 1)
               x1         x2 
  5.0507017  0.1928616 -0.5704941  

Also, there is a theorem stated that " There always exists a value $\lambda$ such that $$MSE(\widehat{\beta_{\lambda}}) < MSE(\widehat{\beta}^{OLS})$$

1
source | link

I hope this helps :

> x1 <- rnorm(30)
> x2<- rnorm(30, mean = x1, sd=0.01)
> y <- rnorm(30, mean = 5+x1+x2)
> cor(x1,x2)
[1] 0.9999316
> fit <- lm(y~x1+x2)$coeff;fit
(Intercept)          x1          x2 
   5.175611    3.859883   -1.452410 
   > 
   > 
   > library(MASS) 
   > lm.ridge(y~x1+x2,lambda = 1)
           x1       x2 
5.171224 1.187841 1.174328 
   > 
   > 
   > 
   > 
   > x1 <- rnorm(30)
   > x2<- runif(30, min = 0, max = 1)
   > cor(x1,x2)
[1] 0.3348988
   > fit <- lm(y~x1+x2)$coeff;fit
(Intercept)          x1          x2 
  5.0691119   0.2017907  -0.6022160 
> lm.ridge(y~x1+x2,lambda = 1)
               x1         x2 
  5.0507017  0.1928616 -0.5704941