Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I have two different implementations of ridge in Matlab. One is simply

(1) $\mathbf x = (\mathbf{A}'\mathbf{A}+\mathbf{I}\lambda)^{-1}\mathbf{A}'\mathbf b$

(as seen here http://en.wikipedia.org/wiki/Ridge_regression), with $\mathbf{I}$ being the identity matrix of size columns($\mathbf{A}$) $\times$ columns($\mathbf{A}$) and

(2) I'm simply calling Matlab's "ridge" with

x = ridge(A, b, lambda)

My problem is that both return different results. (1) returns the results that I want (I know this by comparing results with other people) but why does (2) not return the same results?

My matrix $\mathbf A$ is sparse, it's filled with 1% 1's and 99% 0's. Some columns contain almost no 1's. The biggest difference seems to be that the coefficient for those columns with very few 1's are very close to 0 in (1), but can be quite far from 0 in (2)

Does anyone have any idea why it's different and how I can modify the call in (2) to give the same results as (1)?

Thanks

share|improve this question
2  
My guess: You are calculating (1) on an uncentered and unscaled matrix, whereas the call to ridge (as per the MATLAB documentation) indicates that (as is standard): By default, b is computed after centering and scaling the predictors to have mean 0 and standard deviation 1. The model does not include a constant term, and X should not contain a column of 1s.. – cardinal Feb 17 '12 at 21:46
If the centering and scaling is the the reason, can I modify "A" and "b" so that the Matlab call gives the same result as (1)? – Susie G. Feb 18 '12 at 12:25
Hi, Susie. Yes you can. First if there is an all-constant (nonzero) column in $\mathbf A$, remove it. Now, simply center and scale $\mathbf A$ yourself before making the call to ridge. You shouldn't need to do anything to $\mathbf b$. – cardinal Feb 18 '12 at 17:18
(I have ignored details of sparsity. When centering, the sparsity will be broken. For very big $\mathbf A$ this can introduce problems computationally.) – cardinal Feb 18 '12 at 17:23

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.