Data consists of 40 observations with 4 dimensions and a response-variable.
When doing a ridge regression on my data and plotting the coefficients and coefficient errors (MSE of the ridge coefficients vs. normal linear regression coefficients) as functions of the regularization parameter 'alpha' I get the follow plots:
Can this really be the case? That the coefficient error decreases as the regularization parameter increases? Wouldn't that imply that coefficients with value ~0 is preferable?