Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
2answers
62 views

Weighting features prior to SVM

I'm building an object detector using HOG features and linear SVM. Some of the regions of the object are more "distinctive" so I would like to give more weight to the features extracted from those ...
3
votes
1answer
118 views

How does feature selection work in Random Forest?

I've been trying to improve the performance of my random forest model, and read the following paper on feature selection using random forest (see algorithm in section IV: Overfitting - A. Feature ...
0
votes
0answers
21 views

validation set with ranking variables

i'm working on an approach of feature selection with SVM model and i have some questions about validation , training and test sets. the idea is to rank variables in decreasing order of relevance ...
1
vote
0answers
67 views

How can I extract features from fMRI network connectivity analysis (FSL nets)?

I have a set of 37 fMRI images from mice which are divided into 4 classes (different drug doses applied). My task is to train classifiers (SVM etc.) on this dataset. Of course feature extraction is a ...
0
votes
1answer
144 views

Using bagged ensemble of regression trees, feature selection based on feature importance

I am working on relating aesthetic scores of given images (about 17k training+validation samples and 280 image features) and getting best result using ensemble of CARTs. Beside achieveing a good ...
0
votes
0answers
101 views

Sampling, feature selection and preprocessing in cross validation

To brief my question, I want to clarify the order of parameter tuning and the correctness of the flow in my scheme. In my classification scheme, there are several steps including: SMOTE (Synthetic ...
0
votes
0answers
25 views

Multiinformation (aka Total Correlation) based feature vector selection

I would be happy to discuss the question with an information theory specialist. It is not exactly about the Mutual Information. I use a multiinformation metric (available, for example, in R's ...
0
votes
0answers
23 views

Machine Learning: Feature scaling with new samples

There are a few ways to perform feature scaling on a training set for machine learning. How does this 'scaling' extend to a new sample? To be specific, let's say values of a feature X1 are as follows: ...
0
votes
0answers
34 views

Get important features of n samples

Suppose I have a data frame of [n_samples, m_features] with the corresponding variances of the features [n_features]. The values in my data is between 0 and 1 so the question is: Is there any way to ...
0
votes
0answers
36 views

Using spike-slab to fit log-link GLM Gamma

I am attempting to model the causal impact (using CausalImpact package in R) of a know discrete event on the change in medical expenditures. I have 12 pre and 6 post period observations and upwards of ...
1
vote
0answers
85 views

AIC, model selection and variable scale

In looking at the formula for the AIC=-2*(LL)-2k and the formula for log likelihood, LL=-n/2*log(2*pi) - n/2*log(sse/n) - n/2, I notice that the term with sse is sensitive to the scale of the ...
0
votes
0answers
45 views

Variable reduction techniques

I am researching variable reduction techniques for time series data. Atm I came up with expert judgement, Stepwise Regression (Forward), Stepwise Regression (Backward) and Granger Causality. Any ...
0
votes
0answers
14 views

Feature selection for all categorical classifiers

This is my first question on Cross Validated. I am trying to reduce the dimensionality of my high-dimensional (m = 1.5M) but small-sample size dataset (n = 7K). Characteristic is that all ...
1
vote
1answer
86 views

Feature selection with genetic algorithm in R [closed]

I'm looking for a R-package that does feature selection using a genetic optimization algorithm. I couldn't find one on CRAN and I wonder whether there is a free one. I would be very appreciative for ...
0
votes
1answer
30 views

How to Filter Junk Features Automatically

A data set that is used to build a regression model might contain "junk" fields. For example if I want to build a model of house prices, the field number of rooms and the size of the house are ...
1
vote
0answers
57 views

Interpretation of matrix factorization results

Matrix factorization methods are known to give good results pertaining to problems like movie recommendation. The method reduces the feature space, which is then used for recommendations. For example ...
2
votes
1answer
159 views

Can I Interpret the impact of variables like positive or negative on the model by Random Forest, as I can do by Logistic Regression

I have created a model for prediction of candidates presence or not . I have used Logistic Regression and Random Forest . By Logistic Regression, I got coefficients associated with 100 features and I ...
1
vote
2answers
62 views

How well should features discriminate to build a good classifier from them?

For my (binary) classification problem I'm developing several features and tune them with ROC curves. At some point, I want to combine them with in classifier. How well should the features perform, ...
2
votes
1answer
377 views

Identifying filtered features after feature selection with scikit learn

Here is my Code for feature selection method in Python: ...
1
vote
2answers
128 views

How do I use Lasso and elastic net as feature selectors?

I have a data set with 900,000 rows and 8 features. I want to look at the significance of each feature so that I can evaluate whether the features I add are viable or not. One method I am using after ...
0
votes
0answers
35 views

Normalizing features extracted from image for training a model

I am trying to build a software to classify cells from images taken by a microscope: First, i have a dataset of images of cells to use as training dataset - I have normalized the images and extracted ...
1
vote
0answers
41 views

random forest analysis with random(clustering) varible

My data consists of presence/absence (PA) of a trait in 354 plants collected from 127 collection sites as response, and a set of 25 climatic continuous variables in each site as predictors. The ...
0
votes
1answer
66 views

How to quantify the similarity/difference between groups of selected features?

Since I have a imbalanced data to deal with, I randomly down-sample the negative data to reach balanced data for following feature selection and classification. To test feature's robustness to ...
0
votes
0answers
18 views

Fisherfaces employing PCA to solve singularity problem

In the Fisherfaces paper they use LDA as a feature selection technique to classify faces. We have a $n\times n$ within class matrix $S_w$ and $\text{rank}(S_w) \le N-C$, where $N$ is the number of ...
1
vote
1answer
156 views

Most Parsimonious Elastic Net Model - choosing $\alpha$ and $\lambda$

How do I calculate which Elastic Net model is the most regularized/parsimonious? I am recreating GLMnet in another language as an exercise. I want to do a grid search over several values of alpha and ...
0
votes
0answers
15 views

adding additional variables to a lasso linear model

in my data set there are 100 variables. I use a lasso regularization in my linear model and about 20 variables are non-zero. I use crossvalidation and an extra validation data set to assess ...
1
vote
0answers
119 views

Calculating Mutual Information for feature selection

In order to determine the importance of some individual features coming from labelled time series, I am trying to calculate the Mutual Information (as showed in "Who do you sync you are?: smartphone ...
0
votes
0answers
18 views

doing some modifications in LARS method

Lars method uses 'correlation' in order to select and enter variables into the regression model. in my research work, i am trying to use some other parameters in order to control the variables ...
0
votes
1answer
67 views

How to interpret the relation or interaction between two variables or features in general for classification?

Suppose I have several features like $X_1, X_2, X_3$ etc. In my model, I have to know whether $X_1$ and $X_2$ will have an impact together. I read somewhere we can make a new feature by $( X_1*X_2 )$, ...
1
vote
0answers
142 views

chi square test for large data sets

I use the Chi-square test for feature selection. I use it only when all entries in the contingency table are greater then 5. Is that the correct approach statistically? What happens for example, if ...
5
votes
3answers
1k views

Text Mining: how to cluster texts (e.g. news articles) with artificial intelligence?

I have built some neural networks (MLP (fully-connected), Elman (recurrent)) for different tasks, like playing Pong, classifying handwritten digits and stuff...additionally I tried to build some first ...
1
vote
0answers
53 views

What are the different ways to discretize continuous features?

I'm working on classification problem. I have a set of continuous & discrete features and I'm trying to list my features in their order of significance. Using chi-squared test as a filter & ...
0
votes
0answers
28 views

What kind of feature selection do I need for text mining?

I have a data set of questions belonging to 10 different categories namely (definitions, factoids, abbreviations, fill in the blanks, verbs, numerals, dates, puzzle, etymology and category relation). ...
0
votes
0answers
52 views

How to qualify which features to use in a predictive model?

When building a predictive model, it's well established that picking/building the right features will draw the line between failure and success in the forecasting task. That's said, some people ...
1
vote
1answer
53 views

Any role of redundancy analysis for inclusion of predictors in regression model?

Is there any role of redundancy analysis (for example using the redun() function of the Hmisc package in R) in finding variables to be included for a regression ...
1
vote
1answer
204 views

Which variable relative importance method to use?

Following is a plot from relaimpo package of R which shows relative importance of predictor variables for regression of mpg on all other variables in mtcars dataset. The relative importance is ...
0
votes
0answers
37 views

reliefF vs Information Gain for feature selection

I have a mixed dataset (binary labels) - continuous, ordinary & discrete attributes and I want to perform feature selection. The objective is to understand most important factors influencing the ...
0
votes
0answers
43 views

Significance testing for Feature selection

Feature Selection: Is there a statistical method to identify whether a feature is relevant or not in case of classification problem? I'm reading about filter models, but as per my understanding, ...
1
vote
1answer
103 views

Machine learning tutorials / examples on data sets larger than a terabyte

I am trying to gather a list of practical ML examples / tutorials on more than a terabyte of data. I'm particularly interested in feature extraction from large data sets that involves aggregation (the ...
0
votes
1answer
268 views

Predictions for rpart model require more variables than shown in the classification tree

Using rpart from the caret package, when plotting the final model I get a classification tree that seems fairly simple (6 variables shown in tree). However, when I request the final variables from ...
3
votes
1answer
301 views

What to conclude from this lasso plot (glmnet)

Following is the plot of glmnet with default alpha(1, hence lasso) using mtcars data set in R with mpg as the DV and others as ...
2
votes
0answers
106 views

Generating features: What level of interaction?

I have multi (3) level data indexed by i,j,t. As such, I can generate fixed effects (dummies) for either ij, it, or jt, (and still achieve identification). I can also do i,j,t separately as well. ...
1
vote
1answer
92 views

Features for Object Detection

I want to build a classifier for detecting Airplanes in images. It is important to note that size and shape of the Airplane does not matter in the image. So for training I might simply use the images ...
0
votes
0answers
20 views

spike slab with R

in spike slab regresson, how do we understand the beta coefficients signicant in output in R? I mean, R gets "bma" and "gnet" solutions like below, these coefficients are the significant ones? So it ...
1
vote
0answers
27 views

Initial screening of variables

I recently came across an approach where the long list of potential predictors (around 100+) is screened for its explanatory power and basis this initial screening, a smaller set of predictors is ...
0
votes
0answers
40 views

Prediction - important features from a new element

On my data I did LSI and got a large matrix (>200000 samples, >6000 features, very sparse). I do SVD on it, keeping only 150 dimensions. When I get a new element, do a folding-in, calculate cosine ...
1
vote
2answers
163 views

Is validation set always necessary?

Lets say I did the following steps: Used some separate development set to select some features. Decided a priori to use only one learning algorithm (SVM) with only default parameter values. Trained ...
0
votes
0answers
126 views

Feature extraction based on correlations

I have a small problem regarding feature extraction with correlation. I have divided my question in four parts hoping that somebody can help me. I have a dataset consisting of fMRI images. Each image ...
0
votes
0answers
227 views

Normalization before feature selection such as Fisher score (F-score)

The Fisher score (F-score) for feature selection is defined as in Combining SVMs with Various Feature Selection Strategies. However the paper doesn't mention any preprocessing to feature values such ...
1
vote
0answers
36 views

How to determine if a categorical variable affects an independent variable?

I am looking at the wine dataset from the R FactoMineRpackage to find out whether the categorical variable soil type affects the ...