Hot answers tagged

9

Both models penalise the inclusion of a non-zero coefficient, using a penalty function. LASSO regression penalises in a way that is proportionate to the absolute magnitude of the coefficient, and ridge regression penalises in a way that is proportionate to the square of the coefficient. Neither model penalises inputs in the likelihood function where the ...


3

Let's examine the coefficient estimates of ridge as a function of the OLS estimates. The matrix expression which minimize the loss is $$ \hat{\beta}^{\text {ridge }}=\left(\mathbf{X}^{T} \mathbf{X}+\lambda \mathbf{I}\right)^{-1} \mathbf{X}^{T} \mathbf{y}$$ According to the authors of Elements of Statistical Learning, each element of that vector can be ...


2

For prediction, we are interested in the best outcome and want to include as much information as possible in the model to explain the response, but still without over fitting (don't capture the noise) as we want our model to generalize well to new data. Generally, lower values of the LASSO tuning parameter are needed for prediction. When a group of ...


1

No, it won't be reasonable to use cv.glmnet for cross validation on time series data. A good explanation how CV should look like on time series data can be found here: https://robjhyndman.com/hyndsight/tscv/ In short, a standard CV would skip time points randomly and thus destroy the temporal relationships you want to identify. (I assume that your samples ...


1

Ridge was originally designed for correlated variables, and that's where it's best. Consider examinations to determine a degree. ( Which supposedly is measuring ability) Which do you think is more reliable: taking the average of all the exams or picking a single exam most correlated with ability (if there is one)? Averaging over the different exams removes ...


Only top voted, non community-wiki answers of a minimum length are eligible