Linked Questions

562 votes
11 answers
663k views

What is the difference between test set and validation set?

I found this confusing when I use the neural network toolbox in Matlab. It divided the raw data set into three parts: training set validation set test set I notice in many training or learning ...
xiaohan2012's user avatar
  • 7,169
70 votes
6 answers
19k views

Standard errors for lasso prediction using R

I'm trying to use a LASSO model for prediction, and I need to estimate standard errors. Surely someone has already written a package to do this. But as far as I can see, none of the packages on CRAN ...
Rob Hyndman's user avatar
  • 57.6k
21 votes
2 answers
27k views

Superiority of LASSO over forward selection/backward elimination in terms of the cross validation prediction error of the model

I obtained three reduced models from a original full model using forward selection backward elimination L1 penalization technique (LASSO) For the models obtained using forward selection/backward ...
user41512's user avatar
  • 221
14 votes
2 answers
2k views

Why doesn't collinearity affect the predictions?

I have read in many places that collinearity doesn't affect the predictions. It only affects the coefficient tests and confidence interval. As a result it cannot be used for causal inference but for ...
armen's user avatar
  • 256
5 votes
2 answers
2k views

How can I know If LASSO logistic regression model is good enough to be feature selection tool?

It is known that LASSO can be used for feature selection. How can I know if the model is reliable for that purpose? In general the model's accuracy, R squared and etc, don't bother me because I don't ...
Amit S's user avatar
  • 57
11 votes
4 answers
263 views

How best to estimate regression parameters subject to constraints?

The setting Consider a least squares model for $y$ as a function of $x,$ possibly nonlinear in the parameters. Abstractly this can be expressed as $$y = f(x;\theta) + \varepsilon$$ with the usual ...
whuber's user avatar
  • 329k
9 votes
1 answer
772 views

Cross validation with nonparametric smoothing regressions

When I use regression models I feel leery of defaulting to an assumptions of linear association; instead I like to explore the functional form of relationships between dependent and explanatory ...
Alexis's user avatar
  • 30.3k
3 votes
1 answer
262 views

Do tree based methods like random forest and gradient boosting produce unbiased estimates?

Could anyone point me to literature that discuss properties of tree based estimators? For example, are they unbiased, consistent, maximum likelihood, efficient, etc?
Nandi Subhrangshu's user avatar
2 votes
1 answer
164 views

Coefficient of highly correlated variables under LASSO and ridge

I have been presented with some interesting questions but unfortunately, I am struggling to provide satisfactory answers. The questions are as follows: How will the regression coefficients of two ...
Alex's user avatar
  • 307
3 votes
0 answers
192 views

Why does lasso return unstable features when using the same data?

I am using scikit-learn to shrink my data set having around 800 features. It is a very noisy data (market and economic data) To my best knowledge, lasso returns same features for the same data set. ...
mlee_jordan's user avatar