Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
1answer
13 views

Explained variance in dimensionality reduction

I am new to dimensionality reduction and I am trying to learn different techniques about it. I am noticing that, unlike PCA, many other algorithms do not provide the explained variance of each feature ...
0
votes
0answers
12 views

Feature selection + cross-validation = incorrect classification score?

I'm working on a small dataset (~30) composed of many features (>100). The task consists in selecting the important positive correlated features that can distinguish the positive from the negative ...
0
votes
0answers
27 views

How to determine whether 2 code snippets are functionally same? [migrated]

Given 2 code snippets I want to check whether they are functionally similar or not. By functional similarity I mean that they should yield same output when provided with same input. I am extracting ...
1
vote
1answer
26 views

Most Informative Features with Naive Bayes

Anyone know how to calculate the most informative features where the attributes are normally distributed using Naive Bayes? My understanding, at least if you have binary attributes, is that you ...
0
votes
0answers
20 views

Select exactly k features for regression

I am interested in running online(streaming) regression, and I want to select the features offline. What's a good technique for this? The number of features will be fixed. It will be a small number ...
1
vote
0answers
20 views

How to use RFECV for feature selection and cross validation

I am still very new to machine learning and trying to figure things out myself. I am using SciKit learn and have a data set of tweets with around 20,000 features (n_features=20,000). So far I achieved ...
0
votes
0answers
6 views

How to fill in the missing labels which have more than 1500 levels

I have a question regarding "classification". I have a data set with the target label of more than 1500 nominal categories (there is no way to bucket them as they are just individual strings). The ...
0
votes
0answers
20 views

Sure Independence Screening and variability of the marginal utilities?

I read this interesting article (with Discussion) about marginal screening in ultrahigh-dimensional association studies: J. Fan and J. Lv. Sure independence screening for ultrahigh dimensional ...
1
vote
1answer
37 views

WEKA: Visualize combined trees of random forest classifier

I have a small data set consisting of 385 entries and around 200 attributes. Because I want to apply attribute selection and because of the limited size of my data set, I got the advice to use the ...
1
vote
0answers
14 views

What is the general procedure or general rules for grouping factor levels?

I am attempting to build a predictive (machine-learning) logistic regression model that contains mostly categorical (non-ordinal) variables. As part of a variable selection process I run a Pearson ...
3
votes
2answers
45 views

Feature selection when using cross validation

I have a limited size data set of 385 entries on which I want to run multiple classifiers and compare their performance using the WEKA experimenter. The number of attributes in this data set is ...
0
votes
0answers
23 views

Good filter methods for multiple feature selection for regression problem

Anybody can suggest filter feature selection methods (would be more useful any links to already implemented MATLAB functions/toolboxes) that can rank combinations of features instead of ranking each ...
0
votes
0answers
14 views

Linear mixed models - selection of variables

I have a database with two groups of patients (with and without treatment - retrospective study) with some variables evaluated at two time intervals (at 0 and 3 months) and some variables only at ...
0
votes
1answer
28 views

What statistical test should I run to select “explicative” features in my dataset?

I have a database with more than 500 samples with 22 quantitative features each and I would like to predict a categorical variable (0 or 1). I am trying to fit a logistic regression model and a neural ...
0
votes
0answers
25 views

How to use chi-squared statistics to select features in multi-class classification in R

I have 50k text records which has almost 20k features and 19 class labels. I want to do a multi-class classification in R. I know that for a binary classification, the table and formula below are the ...
0
votes
0answers
17 views

Fusion of two model reduced sensing systems / Comparison of two models

General context: I have a computer vision problem where I take an image sample then: Analyse* it (details omitted) to obtain a vector of values Run PCA (Principle Component Analysis) on it to ...
0
votes
0answers
14 views

Variable selection in high dimensionality

I was wondering what are some techniques for variable selection when there are a large number of variables lets say 1000, and the entire dataset is too large to fit into memory. How would one go ...
0
votes
0answers
29 views

Neural Net in high dimensions for images

I'm trying to build a neural net for a image recognition problem. My images are way too large to build a straight up NN from just the pixels; they are about (1000, 1000) width,height. So naturally i'm ...
1
vote
2answers
199 views

How to account for participants in a study design?

I have a conceptual problem. I want to find out if stress during the day leads to (stronger) teeth grinding (bruxism) at night. I have a number of participants. They will fill in a self-report ...
0
votes
0answers
33 views

Interview Question: What is a statistical method to perform feature selection for large sparse matrices?

I'm just doing some interview preparation for a data science interview and this question came up. I'm familiar with general feature selection methods such as best subset selection, forward stepwise, ...
0
votes
0answers
16 views

Predicting interactions using supervised learning and finding best features

I have a similar problem to this, where I am trying to figure out which features of individual proteins are most important in predicting interactions between proteins. Are there suitable supervised ...
0
votes
0answers
25 views

Variable selection for Arimax model

I have an econometric dataset with ~350 var and 52 observations. In order to pick suitable variables, we ran univariate regressions and picked the top ~30 candidate predictors. Next we ran a variable ...
0
votes
0answers
8 views

How to analyse periodic data obtained from a vibration experiment?

I have data from a vibration experiment on a structure. This data was obtained by attaching an accelerometer to a FFT analyser, which records the data at a high sampling rate (typically in kHz range) ...
1
vote
0answers
14 views

SVM and concatenation of features

For example I train my SVM on 3 features set of different size: Training data size [rows cols]: features1 [nsamples 100] features2 [nsamples 50] features3 [nsamples 128] And for each feature set ...
0
votes
0answers
14 views

Using a stationary data set with exponential smoothing

I am doing time series forecasting and running Holts Method with several variations.(exponential, damped, simple) ...
1
vote
1answer
49 views

Variable selection in time series data

I have an econometric dataset, 50 observations of 350 variables. They include things like GDP, unemployment, interest rates and their transformation such as YoY change, log transform, first ...
1
vote
1answer
24 views

Using Mahalanobis distance for feature selection in NLP

I want build a classifier that classifies sentences into two categories, and for that I have a training set of 1000 labeled sentences. My features consist of a list of about 8000 words, and for each ...
0
votes
0answers
55 views

When feature extracting for an RBF network, how do I measure the “distinctness” or perhaps “uniqueness” of a sample?

I am relatively new to statistics and machine learning so I do not know if I'm always using the right terminology. Please forgive me. I will use Matlab style syntax to explain things as I am using ...
2
votes
0answers
26 views

Random Forest: Class specific feature importance

I'm using the bigrf R-package to analyse a dataset with ca. 50.000 observations x 120 variables, classified into two groups. After growing a forest of 1000 trees, ...
0
votes
0answers
15 views

Categorical Variable Importance: chi-square vs regression model selection techniques

My observations consist of the list of marks students get in various disciplines during the final year exam in university. The only features I know for each student are what books he have read during ...
0
votes
0answers
10 views

Feature Normalization & Learning

I'm working on a cell classifier (as in Biological Cells) using images obtained by microscope. Right now I have about 12 Features written (color,width-height ratio, shape, couple of texture features, ...
0
votes
0answers
24 views

L1-based feature selection, then classification

Does it make sense to use L1-based feature selection to reduce the feature set of a model, then use another L1-based machine learning algorithm to train the model on the selected set of features? For ...
2
votes
0answers
43 views

How do I use weight vector of SVM and logistic regression for feature importance?

I have trained a SVM and logistic regression classifier on my dataset for binary classification. Both classifier provide a weight vector which is of the size of the number of features. I can use this ...
0
votes
0answers
21 views

How to extend independent significance features test to multiple classes?

I have found this post about feature selection using the independent significance features test. There is also an implementation provided. Unfortunately it only works for 2 classes but I have 4 ...
0
votes
1answer
49 views

Standardization before applying ANOVA?

I have a matrix where the rows are the data points (samples) and the columns are the features. It is a multiclass (4 classes) problem. On this data I want to apply machine learning classifiers. But ...
1
vote
1answer
31 views

Validation: Random Forest Features selection

Context: I have a training dataset with 10000 features and i have selected the most important through a Random Forest. I used my subset dataset to train a Neuronal Net. Problem: When i use the ...
4
votes
1answer
44 views

Information gain and mutual information: different or equal?

I'm very confused about the difference between Information gain and mutual information. to make it even more confusing is that I can find both sources defining them as identical and other which ...
2
votes
0answers
22 views

Feature selection of SVM

My question is three-fold In the context of "Kernelized" support vector machines Is variable/feature selection desirable - especially since we regularize the parameter C to prevent overfitting and ...
1
vote
0answers
21 views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the important of each feature for each pair of classes ...
1
vote
2answers
96 views

How to cross validate stepwise logistic regression?

I have a conceptual problem understanding how to cross validate stepwise logistic regression. Every time the training set is divided it is very likely that different features are chosen based on the ...
0
votes
1answer
42 views

In a random forest, is larger %IncMSE better or worse?

Once I have built a (regression) random forest model in R, the call rf$importance provides me with two measures for each predictor variable, ...
0
votes
1answer
34 views

Feature selection in a “Noisy” environment

My first question - This might be a basic question but I have yet to find an answer; when choosing the features for my model, I have encountered certain features which are vectors themselves. (e.g. ...
0
votes
1answer
35 views

Cross validation for feature selection: still possible to overfit?

I would like to find a good pair of predictors out of about 400 available pairs. To do this I am using LOO cross validation. Since there are so many pairs available, don't I run into the issue that ...
0
votes
1answer
45 views

Weighting features prior to SVM

I'm building an object detector using HOG features and linear SVM. Some of the regions of the object are more "distinctive" so I would like to give more weight to the features extracted from those ...
3
votes
1answer
54 views

How does feature selection work in Random Forest?

I've been trying to improve the performance of my random forest model, and read the following paper on feature selection using random forest (see algorithm in section IV: Overfitting - A. Feature ...
0
votes
0answers
16 views

validation set with ranking variables

i'm working on an approach of feature selection with SVM model and i have some questions about validation , training and test sets. the idea is to rank variables in decreasing order of relevance ...
1
vote
0answers
30 views

How can I extract features from fMRI network connectivity analysis (FSL nets)?

I have a set of 37 fMRI images from mice which are divided into 4 classes (different drug doses applied). My task is to train classifiers (SVM etc.) on this dataset. Of course feature extraction is a ...
0
votes
1answer
57 views

Using bagged ensemble of regression trees, feature selection based on feature importance

I am working on relating aesthetic scores of given images (about 17k training+validation samples and 280 image features) and getting best result using ensemble of CARTs. Beside achieveing a good ...
0
votes
0answers
40 views

Sampling, feature selection and preprocessing in cross validation

To brief my question, I want to clarify the order of parameter tuning and the correctness of the flow in my scheme. In my classification scheme, there are several steps including: SMOTE (Synthetic ...
0
votes
0answers
11 views

Multiinformation (aka Total Correlation) based feature vector selection

I would be happy to discuss the question with an information theory specialist. It is not exactly about the Mutual Information. I use a multiinformation metric (available, for example, in R's ...