New answers tagged

1

I wouldn't rely too much on the fact that glmnet includes the c-statistic as a built-in criterion. It also includes accuracy as a criterion for logistic regression, and you will find few on this site who think of that as useful. Frank Harrell himself doesn't think it is useful for discriminating among models, although it can be a very convenient and easily ...


0

@Kolassa has a great mathematical answer. For a more intuitive visual answer here is a picture. I'm doing simple linear regression here with a slope and y-intercept. A population of 17 points are loosely correlated. At random I picked two points and created a regression. In general, 2 points is not enough observations and my regression lines are going ...


0

You could try applying the Elastic Net Regression some times. It combines the Lasso and Ridge regression methods in order to give your feature selection a 'human touch'. This is quite helpful in optimizing your features while preserving the intuition from features in your data.


0

For those trying to figure this out: I have found that there is a great difference between allowing glmnet to calculate $\lambda$, and for when we create a range for it to choose from (grid). Here is an example using "applicants" in the College data set from ISLR # Don't forget to set seed set.seed(1) train <- sample(1:dim(College)[1], 0.75*dim(...


0

PCA while reducing the number of features does not care about the class labels. The only thing that it cares about is preserving the maximum variance which may not always be optimal for classification task. L1-Reg on the other hand pushes those features towards zero that do not have much correlation with the class labels. Hence, L1-Reg strives to reduce the ...


2

The main point is that accuracy is not really "suitable for binary classification problems" despite its frequent use as a criterion in model evaluation. In an important sense there is no single "accuracy" measure as it depends on selection of a particular probability cutoff for assigning class membership. For binary classification this ...


0

I recently created a blog post to compare ridge and lasso using a toy data frame of shark attacks. It helped me understand the behaviors of the algorithms especially when correlated variables are present. Take a look and also see this SO question to explain the shrinkage toward zero.


3

$\newcommand{\x}{\mathbf x}$$\newcommand{\one}{\mathbf 1}$$\newcommand{\X}{\mathbf X}$@kjetil b halvorsen's linked answer explains what's happening, but here's an algebraic answer just for the case of ridge regression (since there's a closed form for the solution). Suppose we have $X\in\mathbb R^{n\times (p+k)}$ as our feature matrix where $$ X = (\...


Top 50 recent answers are included