# Ridge regression via OLS with “phoney data”

*********Okay so I figured out what was wrong! I wasn't centering the date like the lm.ridge function does. However I still cannot reproduce the intercept that lm.ridge gives me.

According to my research you can simulate a ridge regression by adding "phony data" to the end of a normal OLS regression. One of many places that corroborate this notion is this CV thread: Phoney data and ridge regression are the same?

However I fail to replicate the results in R. Here are my three variables:

> test_0
12    34    24    64   746    24    23    42     7     8     3     4    45   675     3     4    34    43  56   674     3     4    54    34    23    34   435    56    56   234   657    89   980     8    76    65 45564    67    76   789

> test_1
34    24    64   746    24    23    42     7     8     3     4    45   675     3     4    34    43    56 674     3     4    54    34    23    34   435    56    56   234   657    89   980     8    76    65 45564  67    76   789     6

> test_2
24    64   746    24    23    42     7     8     3     4    45   675     3     4    34    43    56   674 3     4    54    34    23    34   435    56    56   234  657    89   980     8    76    65 45564    67 76   789     6     5


I then append 2 new rows (for the number of independent vars). To test_0 I append two zeros. To test_1 I append a sqrt(.5) and 0. To test_2 I append a 0 and sqrt(.5)

> a = c(test_0, 0, 0)
> b = c(test_1, (sqrt(.5)), 0)
> c = c(test_2, 0, (sqrt(.5)))


Then I run two models, lm and lm.ridge:

>reg = lm(a~b+c)
>ridge = lm.ridge(test_0~test_1+test_2, lambda=.5)
> reg
Call:
lm(formula = a ~ b + c)

Coefficients:
(Intercept)            b            c
1305.42310     -0.02926     -0.02862

> ridge
test_1        test_2
1374.16801379   -0.03059968   -0.02996396


The coefficients are different but they should be the same. Why is this the case?

I have also tried the above using a lambda of 1 and still get the inconsistency.

• Okay so I figured out what was wrong! I wasn't centering the date like the lm.ridge function does. However I still cannot reproduce the intercept that lm.ridge gives me. – Stevens Sep 18 '15 at 18:11
• You can post this as an answer to help future users. They may even upvote your answer to allow you to accrue reputation. – Sycorax Sep 18 '15 at 19:48
• You should mention in your post what package lm.ridge is from – Glen_b Sep 19 '15 at 0:18

## 1 Answer

The reason my values are off was because the the regular ridge regression method assumes you centralize (standardize) your data in the y vector and X matrix.

If you centralize and then run an OLS with the "phoney data" added in, you get the Ridge Regression betas. The intercept is another story, which I haven't figured out.