Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
1answer
52 views
0
votes
0answers
10 views

Feature selection + cross-validation = incorrect classification score?

I'm working on a small dataset (~30) composed of many features (>100). The task consists in selecting the important positive correlated features that can distinguish the positive from the negative ...
0
votes
0answers
26 views

How to determine whether 2 code snippets are functionally same?

Given 2 code snippets I want to check whether they are functionally similar or not. By functional similarity I mean that they should yield same output when provided with same input. I am extracting ...
14
votes
3answers
564 views

How can SVM 'find' an infinite feature space where linear separation is always possible?

What is the intuition behind the fact that an SVM with a Gaussian Kernel has infinite dimensional feature space?
2
votes
1answer
186 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
1
vote
1answer
26 views

Most Informative Features with Naive Bayes

Anyone know how to calculate the most informative features where the attributes are normally distributed using Naive Bayes? My understanding, at least if you have binary attributes, is that you ...
2
votes
1answer
422 views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
1
vote
1answer
143 views

Justification for feature selection by removing predictors with near zero variance

I have a large number of variables that I'm trying to reduce, and I've stumbled on Kuhn's (2008) suggestion that I eliminate variables with zero or near-zero variance. This makes sense to me, it's ...
0
votes
0answers
20 views

Select exactly k features for regression

I am interested in running online(streaming) regression, and I want to select the features offline. What's a good technique for this? The number of features will be fixed. It will be a small number ...
1
vote
1answer
33 views

WEKA: Visualize combined trees of random forest classifier

I have a small data set consisting of 385 entries and around 200 attributes. Because I want to apply attribute selection and because of the limited size of my data set, I got the advice to use the ...
1
vote
0answers
14 views

How to use RFECV for feature selection and cross validation

I am still very new to machine learning and trying to figure things out myself. I am using SciKit learn and have a data set of tweets with around 20,000 features (n_features=20,000). So far I achieved ...
0
votes
0answers
6 views

How to fill in the missing labels which have more than 1500 levels

I have a question regarding "classification". I have a data set with the target label of more than 1500 nominal categories (there is no way to bucket them as they are just individual strings). The ...
2
votes
1answer
227 views

How to increase the performance of random forest classifier?

I have a text classification task. These are the metrics for different languages at present: class1: 0.6823 class2: 0.7450 class3: 0.66 class4: 0.6719 How can I ...
3
votes
1answer
793 views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
0
votes
0answers
20 views

Sure Independence Screening and variability of the marginal utilities?

I read this interesting article (with Discussion) about marginal screening in ultrahigh-dimensional association studies: J. Fan and J. Lv. Sure independence screening for ultrahigh dimensional ...
2
votes
1answer
127 views

Feature selection and training on the same sample

Is feature selection and training on the same sample a bad idea? I want to emphasize that I am not going to use test set for feature selection. If I use the whole train set for feature selection and ...
1
vote
0answers
13 views

What is the general procedure or general rules for grouping factor levels?

I am attempting to build a predictive (machine-learning) logistic regression model that contains mostly categorical (non-ordinal) variables. As part of a variable selection process I run a Pearson ...
3
votes
2answers
45 views

Feature selection when using cross validation

I have a limited size data set of 385 entries on which I want to run multiple classifiers and compare their performance using the WEKA experimenter. The number of attributes in this data set is ...
0
votes
0answers
22 views

Good filter methods for multiple feature selection for regression problem

Anybody can suggest filter feature selection methods (would be more useful any links to already implemented MATLAB functions/toolboxes) that can rank combinations of features instead of ranking each ...
0
votes
0answers
14 views

Linear mixed models - selection of variables

I have a database with two groups of patients (with and without treatment - retrospective study) with some variables evaluated at two time intervals (at 0 and 3 months) and some variables only at ...
0
votes
0answers
17 views

Fusion of two model reduced sensing systems / Comparison of two models

General context: I have a computer vision problem where I take an image sample then: Analyse* it (details omitted) to obtain a vector of values Run PCA (Principle Component Analysis) on it to ...
0
votes
1answer
28 views

What statistical test should I run to select “explicative” features in my dataset?

I have a database with more than 500 samples with 22 quantitative features each and I would like to predict a categorical variable (0 or 1). I am trying to fit a logistic regression model and a neural ...
0
votes
0answers
25 views

How to use chi-squared statistics to select features in multi-class classification in R

I have 50k text records which has almost 20k features and 19 class labels. I want to do a multi-class classification in R. I know that for a binary classification, the table and formula below are the ...
2
votes
1answer
252 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
4
votes
3answers
1k views

Methods in R or Python to perform feature selection in unsupervised learning

What are the available methods/implementation in R/Python to discard/select unimportant/important features in data? My data does not have labels (unsupervised). The data has ~100 features with mixed ...
0
votes
0answers
14 views

Variable selection in high dimensionality

I was wondering what are some techniques for variable selection when there are a large number of variables lets say 1000, and the entire dataset is too large to fit into memory. How would one go ...
0
votes
1answer
175 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
0
votes
0answers
55 views

When feature extracting for an RBF network, how do I measure the “distinctness” or perhaps “uniqueness” of a sample?

I am relatively new to statistics and machine learning so I do not know if I'm always using the right terminology. Please forgive me. I will use Matlab style syntax to explain things as I am using ...
0
votes
1answer
103 views

Is F test used for feature selection only for features with numerical and continuous domain?

F statistic/test can be used for feature selection, e.g. from http://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.f_classif.html#sklearn.feature_selection.f_classif ANOVA ...
5
votes
2answers
1k views

Is it possible to use kernel PCA for feature selection?

Is it possible to use kernel principal component analysis (kPCA) for Latent Semantic Indexing (LSI) in the same way as PCA is used? I perform LSI in R using the ...
0
votes
0answers
28 views

Neural Net in high dimensions for images

I'm trying to build a neural net for a image recognition problem. My images are way too large to build a straight up NN from just the pixels; they are about (1000, 1000) width,height. So naturally i'm ...
1
vote
2answers
195 views

How to account for participants in a study design?

I have a conceptual problem. I want to find out if stress during the day leads to (stronger) teeth grinding (bruxism) at night. I have a number of participants. They will fill in a self-report ...
4
votes
1answer
44 views

Information gain and mutual information: different or equal?

I'm very confused about the difference between Information gain and mutual information. to make it even more confusing is that I can find both sources defining them as identical and other which ...
0
votes
1answer
93 views

Find entropy in WEKA

I am new in data mining so sorry for asking this kind of silly question. I am working on FAST feature selection algorithm and for that I need to find entropy of each attribute in dataset. But the ...
0
votes
0answers
32 views

Interview Question: What is a statistical method to perform feature selection for large sparse matrices?

I'm just doing some interview preparation for a data science interview and this question came up. I'm familiar with general feature selection methods such as best subset selection, forward stepwise, ...
0
votes
1answer
106 views

combining multiple classifiers common features

Can multiple binary-classifiers be combined to produce a final output if their feature sets have some common elements? How will this influence the accuracy?
0
votes
0answers
16 views

Predicting interactions using supervised learning and finding best features

I have a similar problem to this, where I am trying to figure out which features of individual proteins are most important in predicting interactions between proteins. Are there suitable supervised ...
1
vote
2answers
155 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
0
votes
0answers
25 views

Variable selection for Arimax model

I have an econometric dataset with ~350 var and 52 observations. In order to pick suitable variables, we ran univariate regressions and picked the top ~30 candidate predictors. Next we ran a variable ...
1
vote
0answers
14 views

SVM and concatenation of features

For example I train my SVM on 3 features set of different size: Training data size [rows cols]: features1 [nsamples 100] features2 [nsamples 50] features3 [nsamples 128] And for each feature set ...
0
votes
0answers
8 views

How to analyse periodic data obtained from a vibration experiment?

I have data from a vibration experiment on a structure. This data was obtained by attaching an accelerometer to a FFT analyser, which records the data at a high sampling rate (typically in kHz range) ...
0
votes
0answers
14 views

Using a stationary data set with exponential smoothing

I am doing time series forecasting and running Holts Method with several variations.(exponential, damped, simple) ...
1
vote
1answer
48 views

Variable selection in time series data

I have an econometric dataset, 50 observations of 350 variables. They include things like GDP, unemployment, interest rates and their transformation such as YoY change, log transform, first ...
1
vote
1answer
24 views

Using Mahalanobis distance for feature selection in NLP

I want build a classifier that classifies sentences into two categories, and for that I have a training set of 1000 labeled sentences. My features consist of a list of about 8000 words, and for each ...
2
votes
0answers
25 views

Random Forest: Class specific feature importance

I'm using the bigrf R-package to analyse a dataset with ca. 50.000 observations x 120 variables, classified into two groups. After growing a forest of 1000 trees, ...
0
votes
1answer
590 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
0
votes
0answers
15 views

Categorical Variable Importance: chi-square vs regression model selection techniques

My observations consist of the list of marks students get in various disciplines during the final year exam in university. The only features I know for each student are what books he have read during ...
0
votes
0answers
10 views

Feature Normalization & Learning

I'm working on a cell classifier (as in Biological Cells) using images obtained by microscope. Right now I have about 12 Features written (color,width-height ratio, shape, couple of texture features, ...
2
votes
1answer
292 views

Using MatchIt to match groups in a retrospective analysis

I am interested in using the R package MatchIt to preprocess my data as to obtain matched groups based on a predefined treatment variable. However I am facing a few issues. The first issue is that ...
0
votes
0answers
24 views

L1-based feature selection, then classification

Does it make sense to use L1-based feature selection to reduce the feature set of a model, then use another L1-based machine learning algorithm to train the model on the selected set of features? For ...