I fitted a lasso logistic regression using glmnet. I use a pretty small dataset with only 51 (28/23) observations. I want to compare the model fit of two possible variable combinations.
- Only control variables
- Control variables + linguistic predictors
Both models are comparable regarding explained deviance with best lambdas (1.:17% | 2.:16% dev. explained from null model).
Now I want also compare the mean cross validated error at the best lambdas. Again both models are pretty close (1.: 1.304177 | 2.: 1.324639).
My questions are:
1.) What exactly measures this score? Is it RMSE as measured in linear regression?
2.) From a predictive perspective: Is such a score either good or bad? (I would guess it is not the best predicitve model on earth)
3.) What would a good score look like?