Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
1answer
15 views

Validation: Random Forest Features selection

Context: I have a training dataset with 10000 features and i have selected the most important through a Random Forest. I used my subset dataset to train a Neuronal Net. Problem: When i use the ...
4
votes
0answers
24 views

Information gain and mutual information: different or equal?

I'm very confused about the difference between Information gain and mutual information. to make it even more confusing is that I can find both sources defining them as identical and other which ...
2
votes
0answers
13 views

Feature selection of SVM

My question is three-fold In the context of "Kernelized" support vector machines Is variable/feature selection desirable - especially since we regularize the parameter C to prevent overfitting and ...
0
votes
0answers
15 views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the important of each feature for each pair of classes ...
1
vote
2answers
82 views

How to cross validate stepwise logistic regression?

I have a conceptual problem understanding how to cross validate stepwise logistic regression. Every time the training set is divided it is very likely that different features are chosen based on the ...
0
votes
1answer
26 views

In a random forest, is larger %IncMSE better or worse?

Once I have built a (regression) random forest model in R, the call rf$importance provides me with two measures for each predictor variable, ...
-3
votes
0answers
53 views

How to prove features are independent and can also be vector [closed]

I have samples strings where each one is composed of alphabets and numbers. Using variable 1 i calculated value of each string. similarly i generated a random string and computed its value using ...
0
votes
1answer
31 views

Feature selection in a “Noisy” environment

My first question - This might be a basic question but I have yet to find an answer; when choosing the features for my model, I have encountered certain features which are vectors themselves. (e.g. ...
0
votes
1answer
33 views

Cross validation for feature selection: still possible to overfit?

I would like to find a good pair of predictors out of about 400 available pairs. To do this I am using LOO cross validation. Since there are so many pairs available, don't I run into the issue that ...
0
votes
1answer
33 views
+50

Weighting features prior to SVM

I'm building an object detector using HOG features and linear SVM. Some of the regions of the object are more "distinctive" so I would like to give more weight to the features extracted from those ...
3
votes
1answer
41 views

How does feature selection work in Random Forest?

I've been trying to improve the performance of my random forest model, and read the following paper on feature selection using random forest (see algorithm in section IV: Overfitting - A. Feature ...
0
votes
0answers
14 views

validation set with ranking variables

i'm working on an approach of feature selection with SVM model and i have some questions about validation , training and test sets. the idea is to rank variables in decreasing order of relevance ...
1
vote
0answers
20 views

How can I extract features from fMRI network connectivity analysis (FSL nets)?

I have a set of 37 fMRI images from mice which are divided into 4 classes (different drug doses applied). My task is to train classifiers (SVM etc.) on this dataset. Of course feature extraction is a ...
0
votes
1answer
42 views

Using bagged ensemble of regression trees, feature selection based on feature importance

I am working on relating aesthetic scores of given images (about 17k training+validation samples and 280 image features) and getting best result using ensemble of CARTs. Beside achieveing a good ...
0
votes
0answers
30 views

Sampling, feature selection and preprocessing in cross validation

To brief my question, I want to clarify the order of parameter tuning and the correctness of the flow in my scheme. In my classification scheme, there are several steps including: SMOTE (Synthetic ...
0
votes
0answers
6 views

Multiinformation (aka Total Correlation) based feature vector selection

I would be happy to discuss the question with an information theory specialist. It is not exactly about the Mutual Information. I use a multiinformation metric (available, for example, in R's ...
0
votes
0answers
17 views

Machine Learning: Feature scaling with new samples

There are a few ways to perform feature scaling on a training set for machine learning. How does this 'scaling' extend to a new sample? To be specific, let's say values of a feature X1 are as follows: ...
0
votes
0answers
23 views

Get important features of n samples

Suppose I have a data frame of [n_samples, m_features] with the corresponding variances of the features [n_features]. The values in my data is between 0 and 1 so the question is: Is there any way to ...
0
votes
0answers
13 views

Using spike-slab to fit log-link GLM Gamma

I am attempting to model the causal impact (using CausalImpact package in R) of a know discrete event on the change in medical expenditures. I have 12 pre and 6 post period observations and upwards of ...
0
votes
0answers
26 views

AIC, model selection and variable scale

In looking at the formula for the AIC=-2*(LL)-2k and the formula for log likelihood, LL=-n/2*log(2*pi) - n/2*log(sse/n) - n/2, I notice that the term with sse is sensitive to the scale of the ...
0
votes
0answers
21 views

Variable reduction techniques

I am researching variable reduction techniques for time series data. Atm I came up with expert judgement, Stepwise Regression (Forward), Stepwise Regression (Backward) and Granger Causality. Any ...
0
votes
0answers
10 views

Feature selection for all categorical classifiers

This is my first question on Cross Validated. I am trying to reduce the dimensionality of my high-dimensional (m = 1.5M) but small-sample size dataset (n = 7K). Characteristic is that all ...
1
vote
1answer
54 views

Feature selection with genetic algorithm in R [closed]

I'm looking for a R-package that does feature selection using a genetic optimization algorithm. I couldn't find one on CRAN and I wonder whether there is a free one. I would be very appreciative for ...
0
votes
1answer
28 views

How to Filter Junk Features Automatically

A data set that is used to build a regression model might contain "junk" fields. For example if I want to build a model of house prices, the field number of rooms and the size of the house are ...
0
votes
0answers
33 views

Interpretation of matrix factorization results

Matrix factorization methods are known to give good results pertaining to problems like movie recommendation. The method reduces the feature space, which is then used for recommendations. For example ...
2
votes
1answer
40 views

Can I Interpret the impact of variables like positive or negative on the model by Random Forest, as I can do by Logistic Regression

I have created a model for prediction of candidates presence or not . I have used Logistic Regression and Random Forest . By Logistic Regression, I got coefficients associated with 100 features and I ...
1
vote
2answers
39 views

How well should features discriminate to build a good classifier from them?

For my (binary) classification problem I'm developing several features and tune them with ROC curves. At some point, I want to combine them with in classifier. How well should the features perform, ...
1
vote
1answer
25 views

Identifying filtered features after feature selection with scikit learn

Here is my Code for feature selection method in Python: ...
1
vote
2answers
62 views

How do I use Lasso and elastic net as feature selectors?

I have a data set with 900,000 rows and 8 features. I want to look at the significance of each feature so that I can evaluate whether the features I add are viable or not. One method I am using after ...
0
votes
0answers
14 views

Normalizing features extracted from image for training a model

I am trying to build a software to classify cells from images taken by a microscope: First, i have a dataset of images of cells to use as training dataset - I have normalized the images and extracted ...
1
vote
0answers
24 views

random forest analysis with random(clustering) varible

My data consists of presence/absence (PA) of a trait in 354 plants collected from 127 collection sites as response, and a set of 25 climatic continuous variables in each site as predictors. The ...
0
votes
1answer
36 views

How to quantify the similarity/difference between groups of selected features?

Since I have a imbalanced data to deal with, I randomly down-sample the negative data to reach balanced data for following feature selection and classification. To test feature's robustness to ...
0
votes
0answers
10 views

Fisherfaces employing PCA to solve singularity problem

In the Fisherfaces paper they use LDA as a feature selection technique to classify faces. We have a $n\times n$ within class matrix $S_w$ and $\text{rank}(S_w) \le N-C$, where $N$ is the number of ...
1
vote
1answer
52 views

Most Parsimonious Elastic Net Model - choosing $\alpha$ and $\lambda$

How do I calculate which Elastic Net model is the most regularized/parsimonious? I am recreating GLMnet in another language as an exercise. I want to do a grid search over several values of alpha and ...
0
votes
0answers
12 views

adding additional variables to a lasso linear model

in my data set there are 100 variables. I use a lasso regularization in my linear model and about 20 variables are non-zero. I use crossvalidation and an extra validation data set to assess ...
1
vote
0answers
43 views

Calculating Mutual Information for feature selection

In order to determine the importance of some individual features coming from labelled time series, I am trying to calculate the Mutual Information (as showed in "Who do you sync you are?: smartphone ...
0
votes
0answers
13 views

doing some modifications in LARS method

Lars method uses 'correlation' in order to select and enter variables into the regression model. in my research work, i am trying to use some other parameters in order to control the variables ...
0
votes
1answer
36 views

How to interpret the relation or interaction between two variables or features in general for classification?

Suppose I have several features like $X_1, X_2, X_3$ etc. In my model, I have to know whether $X_1$ and $X_2$ will have an impact together. I read somewhere we can make a new feature by $( X_1*X_2 )$, ...
1
vote
0answers
37 views

chi square test for large data sets

I use the Chi-square test for feature selection. I use it only when all entries in the contingency table are greater then 5. Is that the correct approach statistically? What happens for example, if ...
4
votes
3answers
304 views

Text Mining: how to cluster texts (e.g. news articles) with artificial intelligence?

I have built some neural networks (MLP (fully-connected), Elman (recurrent)) for different tasks, like playing Pong, classifying handwritten digits and stuff...additionally I tried to build some first ...
0
votes
0answers
25 views

What are the different ways to discretize continuous features?

I'm working on classification problem. I have a set of continuous & discrete features and I'm trying to list my features in their order of significance. Using chi-squared test as a filter & ...
0
votes
0answers
22 views

What kind of feature selection do I need for text mining?

I have a data set of questions belonging to 10 different categories namely (definitions, factoids, abbreviations, fill in the blanks, verbs, numerals, dates, puzzle, etymology and category relation). ...
0
votes
0answers
37 views

How to qualify which features to use in a predictive model?

When building a predictive model, it's well established that picking/building the right features will draw the line between failure and success in the forecasting task. That's said, some people ...
1
vote
1answer
34 views

Any role of redundancy analysis for inclusion of predictors in regression model?

Is there any role of redundancy analysis (for example using the redun() function of the Hmisc package in R) in finding variables to be included for a regression ...
1
vote
1answer
66 views

Which variable relative importance method to use?

Following is a plot from relaimpo package of R which shows relative importance of predictor variables for regression of mpg on all other variables in mtcars dataset. The relative importance is ...
0
votes
0answers
12 views

reliefF vs Information Gain for feature selection

I have a mixed dataset (binary labels) - continuous, ordinary & discrete attributes and I want to perform feature selection. The objective is to understand most important factors influencing the ...
0
votes
0answers
22 views

Significance testing for Feature selection

Feature Selection: Is there a statistical method to identify whether a feature is relevant or not in case of classification problem? I'm reading about filter models, but as per my understanding, ...
1
vote
1answer
62 views

Machine learning tutorials / examples on data sets larger than a terabyte

I am trying to gather a list of practical ML examples / tutorials on more than a terabyte of data. I'm particularly interested in feature extraction from large data sets that involves aggregation (the ...
0
votes
1answer
61 views

Predictions for rpart model require more variables than shown in the classification tree

Using rpart from the caret package, when plotting the final model I get a classification tree that seems fairly simple (6 variables shown in tree). However, when I request the final variables from ...
3
votes
1answer
144 views

What to conclude from this lasso plot (glmnet)

Following is the plot of glmnet with default alpha(1, hence lasso) using mtcars data set in R with mpg as the DV and others as ...