Tagged Questions

24 views

Kernel/Basis function design with regularizer

I am solving this problem: $$\sum_i \parallel f(x_i)- y_i\parallel_2^2 + \lambda <\psi f, \psi f>_{L_2}^2$$ where the second part $<\psi f, \psi f>_2^2$ is regularizer using the linear ...
101 views

How big are regularization parameters values?

I wanted to know how big are the regularization parameter values for ridge or lasso. I have seen most of the places generally using values like 0.1 or 0.01 but in some of my experiments the cross ...
127 views

Why does regularization of coefficient magnitude improve the generalization of linear regression?

What is the basic argument upon which ridge and lasso regression are based on? I went through Tikhonov regularization wiki where it was mentioned that In many cases, tikhonov matrix is chosen as ...
97 views

Ridge regression on subset of variables using SVD

I am trying to figure out an algorithm using singular value decomposition to run a modification of ridge regression in which only some of the variables are penalized. I want the output to match the ...
322 views

When does LASSO select correlated predictors?

I'm using the package 'lars' in R with the following code: ...
Following on from my earlier question, the solution to the normal equations for ridge regression is given by: $$\hat{\beta}_\lambda = (X^TX+\lambda I)^{-1}X^Ty$$ Could you offer any guidance for ...
I have a multiple linear regression problem $y=X\beta+\epsilon$. The number of observations $m$ is large, so by the time the data gets to me it's been summarized into: $m$ $X^TX$ $X^Ty$ $y^Ty$ ...