663k views

### What is the difference between test set and validation set?

I found this confusing when I use the neural network toolbox in Matlab. It divided the raw data set into three parts: training set validation set test set I notice in many training or learning ...
• 7,169
19k views

### Standard errors for lasso prediction using R

I'm trying to use a LASSO model for prediction, and I need to estimate standard errors. Surely someone has already written a package to do this. But as far as I can see, none of the packages on CRAN ...
• 57.6k
27k views

### Superiority of LASSO over forward selection/backward elimination in terms of the cross validation prediction error of the model

I obtained three reduced models from a original full model using forward selection backward elimination L1 penalization technique (LASSO) For the models obtained using forward selection/backward ...
• 221
2k views

### Why doesn't collinearity affect the predictions?

I have read in many places that collinearity doesn't affect the predictions. It only affects the coefficient tests and confidence interval. As a result it cannot be used for causal inference but for ...
• 256
2k views

### How can I know If LASSO logistic regression model is good enough to be feature selection tool?

It is known that LASSO can be used for feature selection. How can I know if the model is reliable for that purpose? In general the model's accuracy, R squared and etc, don't bother me because I don't ...
• 57
263 views

### How best to estimate regression parameters subject to constraints?

The setting Consider a least squares model for $y$ as a function of $x,$ possibly nonlinear in the parameters. Abstractly this can be expressed as $$y = f(x;\theta) + \varepsilon$$ with the usual ...
• 329k
772 views

### Cross validation with nonparametric smoothing regressions

When I use regression models I feel leery of defaulting to an assumptions of linear association; instead I like to explore the functional form of relationships between dependent and explanatory ...
• 30.3k
262 views

### Do tree based methods like random forest and gradient boosting produce unbiased estimates?

Could anyone point me to literature that discuss properties of tree based estimators? For example, are they unbiased, consistent, maximum likelihood, efficient, etc?
164 views

### Coefficient of highly correlated variables under LASSO and ridge

I have been presented with some interesting questions but unfortunately, I am struggling to provide satisfactory answers. The questions are as follows: How will the regression coefficients of two ...
• 307