Linked Questions

1
vote
0answers
53 views

How to compare the performance of different feature subsets with the same classifier? [duplicate]

I have a small dataset (55 samples) described by 20 features. I performed a SVM (RBF) approach with cross-validation on 70% of the dataset (training part) and I recorded the AUC (average) for 150 ...
168
votes
6answers
106k views

How to choose a predictive model after k-fold cross-validation?

I am wondering how to choose a predictive model after doing K-fold cross-validation. This may be awkwardly phrased, so let me explain in more detail: whenever I run K-fold cross-validation, I use K ...
54
votes
3answers
31k views

Perform feature normalization before or within model validation?

A common good practice in Machine Learning is to do feature normalization or data standardization of the predictor variables, that's it, center the data substracting the mean and normalize it dividing ...
24
votes
5answers
7k views

Alternatives to classification trees, with better predictive (e.g: CV) performance?

I am looking for an alternative to Classification Trees which might yield better predictive power. The data I am dealing with has factors for both the explanatory and the explained variables. I ...
21
votes
2answers
6k views

Cross Validation (error generalization) after model selection

Note: Case is n>>p I am reading Elements of Statistical Learning and there are various mentions about the "right" way to do cross validation( e.g. page 60, page 245). Specifically, my question is how ...
5
votes
2answers
10k views

Final Model Prediction using K-Fold Cross-Validation and Machine Learning Methods

Similar threads: Feature selection for "final" model when performing cross-validation in machine learning How to choose a predictive model after k-fold cross-validation? My question is ...
18
votes
1answer
2k views

How to build the final model and tune probability threshold after nested cross-validation?

Firstly, apologies for posting a question that has already been discussed at length here, here, here, here, here, and for reheating an old topic. I know @DikranMarsupial has written about this topic ...
3
votes
2answers
1k views

Feature selection using cross validation

I am dealing with a typical $p > n$ problem in the medical field. (typically $p \approx 3700$ and $n \approx 100$ ). The dependent variable is binary (healthy/sick) and features are continuous ...
3
votes
2answers
2k views

Model Tuning and Model Evaluation in Machine Learning

Despite my readings (on stack 1, 2, or in literature (Cawley, 2010; Japkowicz, 2011)), I don't find a clear procedure for tuning and evaluating a model in a classification task. I want to perform a ...
4
votes
2answers
2k views

Cross-validation and feature selection of a multivariate regression

I've been trying to create a multivariate regression model to fit my training data into the prediction of a value. I've put my data into a matrix X with ...
1
vote
1answer
538 views

How can I use synthetic data to validate my classification model?

Through R and based on a microarray gene expression dataset (60 samples in total-30 cancer and 30 control samples) and R package caret, i have performed a feature selection regarding a binary ...
0
votes
1answer
833 views

Forward search feature selection and cross-validation

I've a question regarding forward search for feature selection. Basically, I've found here and here that the procedure is the following: As the procedure suggests, the cross-validation is applied ...
4
votes
1answer
524 views

Is this a correct procedure for feature selection using cross-validation?

I was looking for information about feature selection and crossvalidation, when I found this post: Feature selection for "final" model when performing cross-validation in machine learning. ...
1
vote
1answer
446 views

Feature selection and cross validation

I'm working on a project and I would like to know if the following strategy is good/correct. The input is a dataset with 2.500 features and 1.000 instances. I have to apply a feature selection on ...
3
votes
0answers
550 views

Microarray data: suggestions on Feature selection + Model training scheme?

I have a microarray expression dataset (46 samples, thousands of attributes) and I want to perform feature selection first, and, based on this subset of features (shouldn't be more than 4 or 5, based ...

15 30 50 per page