57 views

### How to compare the performance of different feature subsets with the same classifier? [duplicate]

I have a small dataset (55 samples) described by 20 features. I performed a SVM (RBF) approach with cross-validation on 70% of the dataset (training part) and I recorded the AUC (average) for 150 ...
150k views

### How to choose a predictive model after k-fold cross-validation?

I am wondering how to choose a predictive model after doing K-fold cross-validation. This may be awkwardly phrased, so let me explain in more detail: whenever I run K-fold cross-validation, I use K ...
36k views

### Perform feature normalization before or within model validation?

A common good practice in Machine Learning is to do feature normalization or data standardization of the predictor variables, that's it, center the data substracting the mean and normalize it dividing ...
10k views

### Alternatives to classification trees, with better predictive (e.g: CV) performance?

I am looking for an alternative to Classification Trees which might yield better predictive power. The data I am dealing with has factors for both the explanatory and the explained variables. I ...
7k views

### Cross Validation (error generalization) after model selection

Note: Case is n>>p I am reading Elements of Statistical Learning and there are various mentions about the "right" way to do cross validation( e.g. page 60, page 245). Specifically, my question is how ...
5k views

### How to build the final model and tune probability threshold after nested cross-validation?

Firstly, apologies for posting a question that has already been discussed at length here, here, here, here, here, and for reheating an old topic. I know @DikranMarsupial has written about this topic ...
13k views

### Final Model Prediction using K-Fold Cross-Validation and Machine Learning Methods

Similar threads: Feature selection for "final" model when performing cross-validation in machine learning How to choose a predictive model after k-fold cross-validation? My question is ...
3k views

### Feature selection using cross validation

I am dealing with a typical $p > n$ problem in the medical field. (typically $p \approx 3700$ and $n \approx 100$ ). The dependent variable is binary (healthy/sick) and features are continuous ...
2k views

### Cross-validation and feature selection of a multivariate regression

I've been trying to create a multivariate regression model to fit my training data into the prediction of a value. I've put my data into a matrix X with ...
2k views

### Model Tuning and Model Evaluation in Machine Learning

Despite my readings (on stack 1, 2, or in literature (Cawley, 2010; Japkowicz, 2011)), I don't find a clear procedure for tuning and evaluating a model in a classification task. I want to perform a ...
673 views

### How can I use synthetic data to validate my classification model?

Through R and based on a microarray gene expression dataset (60 samples in total-30 cancer and 30 control samples) and R package caret, i have performed a feature selection regarding a binary ...
565 views

### Student t-tests — please verify if I'm using it correctly for feature selection

I have a $198$-sample dataset containing miRNA types (numerical features) and one categorical feature "Type" with values "Tumor" or "Healthy". ...
948 views

### Forward search feature selection and cross-validation

I've a question regarding forward search for feature selection. Basically, I've found here and here that the procedure is the following: As the procedure suggests, the cross-validation is applied ...