Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

6
votes
1answer
91 views

Why is feature selection important, for classification tasks?

I'm learning about feature selection. I can see why it would be important and useful, for model-building. But let's focus on supervised learning (classification) tasks. Why is feature selection ...
1
vote
1answer
58 views

normalizing before feature extraction for SVM and C 4.5

I'm experimenting with normalization and feature extraction based on Mutual Information before classification for SVM and C4.5. So far I have come to the following conclusions: Feature extraction ...
0
votes
0answers
60 views

Feature Selection with Mutual Information Using “infotheo” package in R

I'm using mutual information for feature selection using "infotheo" for a classification task. A function called "mutinformation" returns a N by N matrix.I know that if the attributes have high value ...
0
votes
0answers
40 views

Threshold for feature selection using gain ratio

I have a dataset with 50 attributes and around 1.2 million tuples and want to perform binary classification. More importantly, my task is to point out major factors which lead to my class label being ...
0
votes
1answer
61 views

In this classification dataset preprocessing, should outliers be removed before/after reducing dimensionality?

I have a classification dataset with 148 input (independent) features, most of which are expected beforehand to be irrelevant. So, at the moment, I am using feature selection methods to discard the ...
0
votes
0answers
18 views

Method for Feature Selection

I am looking for a method that would allow me to find the most influential features from a small dataset (22 biological data points that have varying standard deviations). We have identified 40 ...
0
votes
0answers
33 views

Should I trust the data if the target variable is very loosely correlated with input features?

In my data set, the target variable's correlation with all the feature vectors is less than 0.01. My doubt is whether using these features will affect my predictive model, or should I choose a ...
1
vote
0answers
52 views

Suggestions for Exploratory Data Analysis for high-dimensional regression data?

I have a moderately high-dim dataset (around 20 predictors) that I plan to analyse using Bayesian variable selection techniques. This will be a logistic regression analysis since the output is a ...
0
votes
1answer
48 views

How to cut down on possible predictors?

I am trying to model stock returns with the help of google trends data. As explained in my first question this data is normalised by google so that it is not normally distributed as User EdM kindly ...
0
votes
1answer
59 views

Explained variance in dimensionality reduction

I am new to dimensionality reduction and I am trying to learn different techniques about it. I am noticing that, unlike PCA, many other algorithms do not provide the explained variance of each feature ...
0
votes
0answers
23 views

Feature selection + cross-validation = incorrect classification score?

I'm working on a small dataset (~30) composed of many features (>100). The task consists in selecting the important positive correlated features that can distinguish the positive from the negative ...
1
vote
2answers
67 views

Using correlation to eliminate predictors? [duplicate]

I have 1 dependent variable and 33 independent variables (continuous, categorical & dichotomous). Correlation analyses (2-tailed) show that the DV is only correlated to 7 of the IVs although most ...
1
vote
1answer
62 views

Most Informative Features with Naive Bayes

Anyone know how to calculate the most informative features where the attributes are normally distributed using Naive Bayes? My understanding, at least if you have binary attributes, is that you ...
0
votes
0answers
23 views

Select exactly k features for regression

I am interested in running online(streaming) regression, and I want to select the features offline. What's a good technique for this? The number of features will be fixed. It will be a small number ...
1
vote
0answers
125 views

How to use RFECV for feature selection and cross validation

I am still very new to machine learning and trying to figure things out myself. I am using SciKit learn and have a data set of tweets with around 20,000 features (n_features=20,000). So far I achieved ...
0
votes
0answers
13 views

How to fill in the missing labels which have more than 1500 levels

I have a question regarding "classification". I have a data set with the target label of more than 1500 nominal categories (there is no way to bucket them as they are just individual strings). The ...
0
votes
0answers
28 views

Sure Independence Screening and variability of the marginal utilities?

I read this interesting article (with Discussion) about marginal screening in ultrahigh-dimensional association studies: J. Fan and J. Lv. Sure independence screening for ultrahigh dimensional ...
2
votes
2answers
457 views

WEKA: Visualize combined trees of random forest classifier

I have a small data set consisting of 385 entries and around 200 attributes. Because I want to apply attribute selection and because of the limited size of my data set, I got the advice to use the ...
1
vote
0answers
21 views

What is the general procedure or general rules for grouping factor levels?

I am attempting to build a predictive (machine-learning) logistic regression model that contains mostly categorical (non-ordinal) variables. As part of a variable selection process I run a Pearson ...
3
votes
2answers
95 views

Feature selection when using cross validation

I have a limited size data set of 385 entries on which I want to run multiple classifiers and compare their performance using the WEKA experimenter. The number of attributes in this data set is ...
0
votes
0answers
72 views

Good filter methods for multiple feature selection for regression problem

Anybody can suggest filter feature selection methods (would be more useful any links to already implemented MATLAB functions/toolboxes) that can rank combinations of features instead of ranking each ...
0
votes
0answers
17 views

Linear mixed models - selection of variables

I have a database with two groups of patients (with and without treatment - retrospective study) with some variables evaluated at two time intervals (at 0 and 3 months) and some variables only at ...
0
votes
1answer
37 views

What statistical test should I run to select “explicative” features in my dataset?

I have a database with more than 500 samples with 22 quantitative features each and I would like to predict a categorical variable (0 or 1). I am trying to fit a logistic regression model and a neural ...
1
vote
0answers
85 views

How to use chi-squared statistics to select features in multi-class classification in R

I have 50k text records which has almost 20k features and 19 class labels. I want to do a multi-class classification in R. I know that for a binary classification, the table and formula below are the ...
0
votes
0answers
22 views

Fusion of two model reduced sensing systems / Comparison of two models

General context: I have a computer vision problem where I take an image sample then: Analyse* it (details omitted) to obtain a vector of values Run PCA (Principle Component Analysis) on it to ...
0
votes
0answers
17 views

Variable selection in high dimensionality

I was wondering what are some techniques for variable selection when there are a large number of variables lets say 1000, and the entire dataset is too large to fit into memory. How would one go ...
0
votes
0answers
40 views

Neural Net in high dimensions for images

I'm trying to build a neural net for a image recognition problem. My images are way too large to build a straight up NN from just the pixels; they are about (1000, 1000) width,height. So naturally i'm ...
1
vote
2answers
234 views

How to account for participants in a study design?

I have a conceptual problem. I want to find out if stress during the day leads to (stronger) teeth grinding (bruxism) at night. I have a number of participants. They will fill in a self-report ...
0
votes
0answers
67 views

Interview Question: What is a statistical method to perform feature selection for large sparse matrices?

I'm just doing some interview preparation for a data science interview and this question came up. I'm familiar with general feature selection methods such as best subset selection, forward stepwise, ...
0
votes
0answers
21 views

Predicting interactions using supervised learning and finding best features

I have a similar problem to this, where I am trying to figure out which features of individual proteins are most important in predicting interactions between proteins. Are there suitable supervised ...
0
votes
0answers
58 views

Variable selection for Arimax model

I have an econometric dataset with ~350 var and 52 observations. In order to pick suitable variables, we ran univariate regressions and picked the top ~30 candidate predictors. Next we ran a variable ...
0
votes
0answers
12 views

How to analyse periodic data obtained from a vibration experiment?

I have data from a vibration experiment on a structure. This data was obtained by attaching an accelerometer to a FFT analyser, which records the data at a high sampling rate (typically in kHz range) ...
1
vote
0answers
20 views

SVM and concatenation of features

For example I train my SVM on 3 features set of different size: Training data size [rows cols]: features1 [nsamples 100] features2 [nsamples 50] features3 [nsamples 128] And for each feature set ...
0
votes
0answers
52 views

Using a stationary data set with exponential smoothing

I am doing time series forecasting and running Holts Method with several variations.(exponential, damped, simple) ...
1
vote
1answer
132 views

Variable selection in time series data

I have an econometric dataset, 50 observations of 350 variables. They include things like GDP, unemployment, interest rates and their transformation such as YoY change, log transform, first ...
1
vote
1answer
37 views

Using Mahalanobis distance for feature selection in NLP

I want build a classifier that classifies sentences into two categories, and for that I have a training set of 1000 labeled sentences. My features consist of a list of about 8000 words, and for each ...
0
votes
0answers
63 views

When feature extracting for an RBF network, how do I measure the “distinctness” or perhaps “uniqueness” of a sample?

I am relatively new to statistics and machine learning so I do not know if I'm always using the right terminology. Please forgive me. I will use Matlab style syntax to explain things as I am using ...
3
votes
0answers
68 views

Random Forest: Class specific feature importance

I'm using the bigrf R-package to analyse a dataset with ca. 50.000 observations x 120 variables, classified into two groups. After growing a forest of 1000 trees, ...
0
votes
0answers
24 views

Categorical Variable Importance: chi-square vs regression model selection techniques

My observations consist of the list of marks students get in various disciplines during the final year exam in university. The only features I know for each student are what books he have read during ...
0
votes
0answers
12 views

Feature Normalization & Learning

I'm working on a cell classifier (as in Biological Cells) using images obtained by microscope. Right now I have about 12 Features written (color,width-height ratio, shape, couple of texture features, ...
0
votes
0answers
34 views

L1-based feature selection, then classification

Does it make sense to use L1-based feature selection to reduce the feature set of a model, then use another L1-based machine learning algorithm to train the model on the selected set of features? For ...
2
votes
0answers
104 views

How do I use weight vector of SVM and logistic regression for feature importance?

I have trained a SVM and logistic regression classifier on my dataset for binary classification. Both classifier provide a weight vector which is of the size of the number of features. I can use this ...
0
votes
0answers
29 views

How to extend independent significance features test to multiple classes?

I have found this post about feature selection using the independent significance features test. There is also an implementation provided. Unfortunately it only works for 2 classes but I have 4 ...
0
votes
1answer
79 views

Standardization before applying ANOVA?

I have a matrix where the rows are the data points (samples) and the columns are the features. It is a multiclass (4 classes) problem. On this data I want to apply machine learning classifiers. But ...
2
votes
1answer
52 views

Validation: Random Forest Features selection

Context: I have a training dataset with 10000 features and i have selected the most important through a Random Forest. I used my subset dataset to train a Neuronal Net. Problem: When i use the ...
4
votes
1answer
93 views

Information gain and mutual information: different or equal?

I'm very confused about the difference between Information gain and mutual information. to make it even more confusing is that I can find both sources defining them as identical and other which ...
2
votes
0answers
46 views

Feature selection of SVM

My question is three-fold In the context of "Kernelized" support vector machines Is variable/feature selection desirable - especially since we regularize the parameter C to prevent overfitting and ...
1
vote
0answers
80 views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the important of each feature for each pair of classes ...
1
vote
2answers
140 views

How to cross validate stepwise logistic regression?

I have a conceptual problem understanding how to cross validate stepwise logistic regression. Every time the training set is divided it is very likely that different features are chosen based on the ...
0
votes
1answer
630 views

In a random forest, is larger %IncMSE better or worse?

Once I have built a (regression) random forest model in R, the call rf$importance provides me with two measures for each predictor variable, ...