Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
1answer
13 views

Validation: Random Forest Features selection

Context: I have a training dataset with 10000 features and i have selected the most important through a Random Forest. I used my subset dataset to train a Neuronal Net. Problem: When i use the ...
4
votes
0answers
24 views

Information gain and mutual information: different or equal?

I'm very confused about the difference between Information gain and mutual information. to make it even more confusing is that I can find both sources defining them as identical and other which ...
7
votes
1answer
1k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
2
votes
0answers
13 views

Feature selection of SVM

My question is three-fold In the context of "Kernelized" support vector machines Is variable/feature selection desirable - especially since we regularize the parameter C to prevent overfitting and ...
0
votes
0answers
15 views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the important of each feature for each pair of classes ...
1
vote
2answers
82 views

How to cross validate stepwise logistic regression?

I have a conceptual problem understanding how to cross validate stepwise logistic regression. Every time the training set is divided it is very likely that different features are chosen based on the ...
1
vote
3answers
215 views

Evaluating features and similarity measures

I am currently developing a classifier, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
-3
votes
0answers
53 views

How to prove features are independent and can also be vector [closed]

I have samples strings where each one is composed of alphabets and numbers. Using variable 1 i calculated value of each string. similarly i generated a random string and computed its value using ...
0
votes
1answer
31 views
+50

Weighting features prior to SVM

I'm building an object detector using HOG features and linear SVM. Some of the regions of the object are more "distinctive" so I would like to give more weight to the features extracted from those ...
2
votes
1answer
182 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
0
votes
1answer
25 views

In a random forest, is larger %IncMSE better or worse?

Once I have built a (regression) random forest model in R, the call rf$importance provides me with two measures for each predictor variable, ...
0
votes
1answer
31 views

Feature selection in a “Noisy” environment

My first question - This might be a basic question but I have yet to find an answer; when choosing the features for my model, I have encountered certain features which are vectors themselves. (e.g. ...
2
votes
1answer
339 views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
4
votes
3answers
12k views

Feature Selection Packages in R, which do both regression and classification

I am very new to R. I am learning machine learning right now. Very sorry, if this question appears to be very basic. I am trying to find a good feature selection package in R. I went through Boruta ...
0
votes
1answer
33 views

Cross validation for feature selection: still possible to overfit?

I would like to find a good pair of predictors out of about 400 available pairs. To do this I am using LOO cross validation. Since there are so many pairs available, don't I run into the issue that ...
3
votes
1answer
688 views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
3
votes
1answer
874 views

Step-wise feature selection with caret

can anyone direct me to a package/commands in R for performing step-wise feature selection, preferably using the caret package. I have already used linear ...
3
votes
1answer
41 views

How does feature selection work in Random Forest?

I've been trying to improve the performance of my random forest model, and read the following paper on feature selection using random forest (see algorithm in section IV: Overfitting - A. Feature ...
10
votes
3answers
10k views

Using principal component analysis (PCA) for feature selection

I'm new to feature selection and I was wondering how you would use PCA to perform feature selection. Does PCA compute a relative score for each input variable that you can use to filter out ...
0
votes
0answers
14 views

validation set with ranking variables

i'm working on an approach of feature selection with SVM model and i have some questions about validation , training and test sets. the idea is to rank variables in decreasing order of relevance ...
2
votes
1answer
250 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
2
votes
1answer
182 views

How to increase the performance of random forest classifier?

I have a text classification task. These are the metrics for different languages at present: class1: 0.6823 class2: 0.7450 class3: 0.66 class4: 0.6719 How can I ...
2
votes
1answer
121 views

Feature selection and training on the same sample

Is feature selection and training on the same sample a bad idea? I want to emphasize that I am not going to use test set for feature selection. If I use the whole train set for feature selection and ...
1
vote
0answers
19 views

How can I extract features from fMRI network connectivity analysis (FSL nets)?

I have a set of 37 fMRI images from mice which are divided into 4 classes (different drug doses applied). My task is to train classifiers (SVM etc.) on this dataset. Of course feature extraction is a ...
0
votes
1answer
148 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
0
votes
1answer
42 views

Using bagged ensemble of regression trees, feature selection based on feature importance

I am working on relating aesthetic scores of given images (about 17k training+validation samples and 280 image features) and getting best result using ensemble of CARTs. Beside achieveing a good ...
1
vote
1answer
93 views

Justification for feature selection by removing predictors with near zero variance

I have a large number of variables that I'm trying to reduce, and I've stumbled on Kuhn's (2008) suggestion that I eliminate variables with zero or near-zero variance. This makes sense to me, it's ...
1
vote
2answers
209 views

Feature selection before neural network classification

I have a training set of 87 samples and 9480 variables. My predictors are continuous and my response variable is binary. I'd like to use the caret package in R to tune a neural network classification ...
0
votes
1answer
82 views

Find entropy in WEKA

I am new in data mining so sorry for asking this kind of silly question. I am working on FAST feature selection algorithm and for that I need to find entropy of each attribute in dataset. But the ...
0
votes
0answers
30 views

Sampling, feature selection and preprocessing in cross validation

To brief my question, I want to clarify the order of parameter tuning and the correctness of the flow in my scheme. In my classification scheme, there are several steps including: SMOTE (Synthetic ...
0
votes
0answers
6 views

Multiinformation (aka Total Correlation) based feature vector selection

I would be happy to discuss the question with an information theory specialist. It is not exactly about the Mutual Information. I use a multiinformation metric (available, for example, in R's ...
0
votes
0answers
17 views

Machine Learning: Feature scaling with new samples

There are a few ways to perform feature scaling on a training set for machine learning. How does this 'scaling' extend to a new sample? To be specific, let's say values of a feature X1 are as follows: ...
0
votes
0answers
23 views

Get important features of n samples

Suppose I have a data frame of [n_samples, m_features] with the corresponding variances of the features [n_features]. The values in my data is between 0 and 1 so the question is: Is there any way to ...
0
votes
0answers
13 views

Using spike-slab to fit log-link GLM Gamma

I am attempting to model the causal impact (using CausalImpact package in R) of a know discrete event on the change in medical expenditures. I have 12 pre and 6 post period observations and upwards of ...
2
votes
1answer
272 views

Using MatchIt to match groups in a retrospective analysis

I am interested in using the R package MatchIt to preprocess my data as to obtain matched groups based on a predefined treatment variable. However I am facing a few issues. The first issue is that ...
2
votes
2answers
242 views

How to calculate number of features based on image resolution?

Just covered Andrew Ng's Non-linear Hypothesis of Neural Netowrks, and we had a multiple choice question for determining number of features for an image of resolution 100x100 of grescale intensities. ...
9
votes
5answers
13k views

Using LASSO from lars (or glmnet) package in R for variable selection

Sorry if this question comes across a little basic. I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
1
vote
2answers
151 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
0
votes
1answer
550 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
0
votes
0answers
26 views

AIC, model selection and variable scale

In looking at the formula for the AIC=-2*(LL)-2k and the formula for log likelihood, LL=-n/2*log(2*pi) - n/2*log(sse/n) - n/2, I notice that the term with sse is sensitive to the scale of the ...
0
votes
0answers
21 views

Variable reduction techniques

I am researching variable reduction techniques for time series data. Atm I came up with expert judgement, Stepwise Regression (Forward), Stepwise Regression (Backward) and Granger Causality. Any ...
0
votes
0answers
10 views

Feature selection for all categorical classifiers

This is my first question on Cross Validated. I am trying to reduce the dimensionality of my high-dimensional (m = 1.5M) but small-sample size dataset (n = 7K). Characteristic is that all ...
0
votes
1answer
79 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
1
vote
1answer
54 views

Feature selection with genetic algorithm in R [closed]

I'm looking for a R-package that does feature selection using a genetic optimization algorithm. I couldn't find one on CRAN and I wonder whether there is a free one. I would be very appreciative for ...
4
votes
1answer
79 views

Placement of earlier features in more complex features in CNN

I'm trying to understand convolutional neural networks better. I've been doing different tutorials, but there are some basics concerning how the hidden units represents features that I really would ...
1
vote
1answer
25 views
0
votes
0answers
33 views

Interpretation of matrix factorization results

Matrix factorization methods are known to give good results pertaining to problems like movie recommendation. The method reduces the feature space, which is then used for recommendations. For example ...
0
votes
1answer
28 views

How to Filter Junk Features Automatically

A data set that is used to build a regression model might contain "junk" fields. For example if I want to build a model of house prices, the field number of rooms and the size of the house are ...
2
votes
1answer
40 views

Can I Interpret the impact of variables like positive or negative on the model by Random Forest, as I can do by Logistic Regression

I have created a model for prediction of candidates presence or not . I have used Logistic Regression and Random Forest . By Logistic Regression, I got coefficients associated with 100 features and I ...
3
votes
1answer
87 views

Feature selection: permutation test Vs deleting a variable

In feature selection for predictive models, it is usually applied a permutation test. In this test, all the values of one variable are randomly permuted and the prediction accuracy is extracted for ...