Linked Questions

1
vote
0answers
57 views

How to compare the performance of different feature subsets with the same classifier? [duplicate]

I have a small dataset (55 samples) described by 20 features. I performed a SVM (RBF) approach with cross-validation on 70% of the dataset (training part) and I recorded the AUC (average) for 150 ...
221
votes
6answers
150k views

How to choose a predictive model after k-fold cross-validation?

I am wondering how to choose a predictive model after doing K-fold cross-validation. This may be awkwardly phrased, so let me explain in more detail: whenever I run K-fold cross-validation, I use K ...
61
votes
4answers
36k views

Perform feature normalization before or within model validation?

A common good practice in Machine Learning is to do feature normalization or data standardization of the predictor variables, that's it, center the data substracting the mean and normalize it dividing ...
24
votes
5answers
10k views

Alternatives to classification trees, with better predictive (e.g: CV) performance?

I am looking for an alternative to Classification Trees which might yield better predictive power. The data I am dealing with has factors for both the explanatory and the explained variables. I ...
23
votes
2answers
7k views

Cross Validation (error generalization) after model selection

Note: Case is n>>p I am reading Elements of Statistical Learning and there are various mentions about the "right" way to do cross validation( e.g. page 60, page 245). Specifically, my question is how ...
29
votes
3answers
5k views

How to build the final model and tune probability threshold after nested cross-validation?

Firstly, apologies for posting a question that has already been discussed at length here, here, here, here, here, and for reheating an old topic. I know @DikranMarsupial has written about this topic ...
5
votes
2answers
13k views

Final Model Prediction using K-Fold Cross-Validation and Machine Learning Methods

Similar threads: Feature selection for "final" model when performing cross-validation in machine learning How to choose a predictive model after k-fold cross-validation? My question is ...
4
votes
2answers
3k views

Feature selection using cross validation

I am dealing with a typical $p > n$ problem in the medical field. (typically $p \approx 3700$ and $n \approx 100$ ). The dependent variable is binary (healthy/sick) and features are continuous ...
5
votes
2answers
2k views

Cross-validation and feature selection of a multivariate regression

I've been trying to create a multivariate regression model to fit my training data into the prediction of a value. I've put my data into a matrix X with ...
3
votes
2answers
2k views

Model Tuning and Model Evaluation in Machine Learning

Despite my readings (on stack 1, 2, or in literature (Cawley, 2010; Japkowicz, 2011)), I don't find a clear procedure for tuning and evaluating a model in a classification task. I want to perform a ...
1
vote
1answer
673 views

How can I use synthetic data to validate my classification model?

Through R and based on a microarray gene expression dataset (60 samples in total-30 cancer and 30 control samples) and R package caret, i have performed a feature selection regarding a binary ...
1
vote
1answer
565 views

Student t-tests — please verify if I'm using it correctly for feature selection

I have a $198$-sample dataset containing miRNA types (numerical features) and one categorical feature "Type" with values "Tumor" or "Healthy". ...
0
votes
1answer
948 views

Forward search feature selection and cross-validation

I've a question regarding forward search for feature selection. Basically, I've found here and here that the procedure is the following: As the procedure suggests, the cross-validation is applied ...
4
votes
1answer
547 views

Is this a correct procedure for feature selection using cross-validation?

I was looking for information about feature selection and crossvalidation, when I found this post: Feature selection for "final" model when performing cross-validation in machine learning. ...
1
vote
1answer
478 views

Feature selection and cross validation

I'm working on a project and I would like to know if the following strategy is good/correct. The input is a dataset with 2.500 features and 1.000 instances. I have to apply a feature selection on ...

15 30 50 per page