Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
2answers
81 views

Choosing the best featureset for prediction

I have this N sets of features F each with $F_i$ number of features. All the feature sets have 20000 examples and we have 20,000 labels for them. Lets say feature set $F_1$ has 10 features and ...
6
votes
1answer
840 views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
1
vote
1answer
111 views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
1
vote
1answer
62 views

(Automated) feature selection in multiple regression with categorical variables

I need a general guide on what are the appropriate approaches to automated feature selection in multiple regression with categorical variables. In my case, I have several numeric and categorical ...
1
vote
0answers
10 views

Why normalized feature weights for linear regression are bad feature importance predictors

I am trying to interpret a linear regression model. I assumed using absolute value of feature weight coefficients as indicators of influence of input variable onto output variable. However, it seems ...
1
vote
1answer
72 views

Feature selection for pattern mining

I must find frequent patterns in temporal data, using a method that was imposed to me. This tool has problems handling these data: processing is long and takes a lot of memory. So, I would like to ...
0
votes
1answer
111 views

How to find important features in this problem

I was just thinking how ML techniques can be applied in the retail industry. Suppose we have data from a retailer who deals with apparel and cloth in this format and for each item there are ...
1
vote
1answer
57 views

Strategy for Analyzing Data

I have been learning about Machine Learning (via Udacity) and Statistics (via Coursera) the past few months and trying to figure out a good way to combine them for a general approach to explaining ...
0
votes
1answer
390 views

What is a good Gini decrease cutoff for feature inclusion based upon random forests?

I am using random forests to try and determine variable importance as part of feature selection for a model I'm working on, and while I can get ranked variable importance by mean decrease in Gini from ...
2
votes
1answer
29 views

Feature Selection: Information Gain VS Mutual Information

Setting: Multi-class classification problem with discrete nominal features. There are many references mentioning the use of IG(Information Gain) and ...
0
votes
0answers
13 views

Feature selection when bagging trees/random forest

I want to get a better understanding of feature selection and how the number of features affect performance when bagging trees. I am using Matlab's treebagger and I ...
1
vote
1answer
102 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
0
votes
1answer
6 views

Select the most confident variable that has two features

Suppose now I have a group of students, and for each student two measurements are given: one is the height of the student and the other is the weight of the student. Then my question is how I can ...
1
vote
1answer
205 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
2
votes
1answer
155 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
0
votes
0answers
21 views

Generate mixture model from data with features

I want to build a mixture model from my data, but using features of my data to calculate each component in the model. The data: For each point I have 34 associated features. Each feature is a boolean ...
4
votes
3answers
367 views

How to handle high dimensional feature vector in probability graph model?

I was doing some NLP related stuff which involves training a hidden Markov model, and use the model to segment sentences. For every sentence, I translate the tokens into feature vectors. The features ...
2
votes
3answers
121 views

Methods in R or Python to perform feature selection in unsupervised learning

What are the available methods/implementation in R/Python to discard/select unimportant/important features in data? My data does not have labels (unsupervised). The data has ~100 features with mixed ...
4
votes
2answers
804 views

Is it possible to use kernel PCA for feature selection?

Is it possible to use kernel principal component analysis (kPCA) for Latent Semantic Indexing (LSI) in the same way as PCA is used? I perform LSI in R using the prcomp PCA function and extract the ...
0
votes
0answers
8 views

Caret: customizing feature selection using matrix-wise operations [migrated]

Short question: is it possible to use matrix-wise operations in caretSBF$score function? Motivation: When working with big matrices in R, operations that work natively matrix-wise [e.g. rowMeans(X) ...
0
votes
1answer
26 views

Use fitted value from regression on subset of features as independent variable

I am working with a relatively large data set with 2K columns and many variables can be grouped together (a logistic regression). So I am thinking can I use fitted value from regression on subset of ...
0
votes
1answer
15 views

Weka java API: Attribute Selection and Cross Validation

Is there a way to perform Attirbute selection(aka feature selection) (regardless of method) only for the training dataset before passing data for Cross Validation ? I currently think that the only ...
3
votes
2answers
77 views

Advice for interpolating a model

I'm new in Stack Exchange, so I hope no to be off topic. I'm also new in bioinformatics and I was asked to perform an analysis. Briefly, I have a dataset of 29 cell lines and the IC50 values of a test ...
9
votes
1answer
2k views

How do you select variables in a regression model?

The traditional approach to variable selection is to find variables that contribute the most to predicting a new response. Recently I learned of an alternative to this. In modeling variables that ...
0
votes
0answers
16 views

Feature/Variable selection to accompany mixed models?

I am trying to conduct an exploratory/data mining analysis to discover what socioeconomic factors best predict grade-school performance in children. I have a dataset with about 50000 ...
4
votes
1answer
2k views

Information gain as a feature selection for 3-class classification problem

I am facing a sentiment analysis task where I am using Naive Bayes to classify documents as Positive, Negative or Neutral. I have thought of using Information Gain as my filter for feature selection. ...
1
vote
2answers
143 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
0
votes
0answers
19 views

$\chi^2$ test vs F-test in feature selection

In the context of feature selection for classification, does it make sense to use one filter based on $\chi^2$ test and the other one based on F-test? Or they are "interchangeable"?
0
votes
1answer
29 views

mtry tuning given by caret higher than the number of predictors

According to this discussion, it seems that the train function of the caret package returns a ...
0
votes
0answers
19 views

Determine which variable or variables is/are the most efficient to predict the outcome

I have a small dataset (n=74) with a +/- 50 variables, not the best data but I have to work with it. The variables are used to select a product. I want to determine which variable or variables is/are ...
0
votes
0answers
10 views

Unsupervised feature learning from raw text as a previous step for clasification?

I have a corpus of 2500 opinions, is it posible to use scikit´s restricted boltzmann machine implementation to extract a feature vector as a previous step to a classification task?. What aproach do i ...
0
votes
1answer
128 views

Recursive feature elimination with only two classes

Recursive feature elimination (RFE) is a feature-selection strategy. It performs in two nested levels of cross-validation. First it tries to divide the training set into N folds. RFE puts one fold ...
0
votes
1answer
46 views

Using topic words generated by LDA to represent a document

I want to do document classification by representing each document as a set of features. I know that there are many ways: BOW, TFIDF, ... I want to use Latent Dirichlet Allocation (LDA) to extract ...
0
votes
1answer
30 views

Feature Importance in each fold and repeat after repeated cross validation in caret

this is my first post on Cross Validated so I apologize in advance if I'm not yet familiar with any conventions regarding forum posts. Currently, I'm working on a feature selection task using elastic ...
1
vote
1answer
138 views

Using principal component analysis (PCA) for feature selection in regression [duplicate]

I have a dataset $D$ made of $m$ samples and $n$ features with $n \gg m$. For each sample I have a score $s$ which I would like to be able to predict. As the number of features is very high (compared ...
6
votes
2answers
5k views

Using principal component analysis (PCA) for feature selection

I'm new to feature selection and I was wondering how you would use PCA to perform feature selection. Does PCA compute a relative score for each input variable that you can use to filter out ...
4
votes
4answers
2k views

Term frequency/inverse document frequency (TF/IDF): weighting

I've got a dataset which represents 1000 documents and all the words that appear in it. So the rows represent the documents and the columns represent the words. So for example, the value in cell ...
0
votes
1answer
73 views

Feature selection and training on the same sample

Is feature selection and training on the same sample a bad idea? I want to emphasize that I am not going to use test set for feature selection. If I use the whole train set for feature selection and ...
0
votes
1answer
142 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
2
votes
3answers
106 views

What if Lasso selects transformed terms but not untransformed terms

Suppose I have standard normal features $X_i \in \{X_i : i \in \{1,...,1000\}\}$. I extend this set of predictors with transformations as follows: $\{X_i,X_i^2,X_iI(X_i > 0) : i \in ...
3
votes
1answer
453 views

Can we use random forest for classification in combination with distance matrix between classes?

With a colleague, we are working on a dataset containing ~5000 continuous variables for 120 individuals belonging to 8 classes. We want to estimate the relative importance of each variable to explain ...
11
votes
5answers
385 views

What can cause PCA to worsen results of a classifier?

I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against ...
4
votes
4answers
6k views

Using LASSO from lars (or glmnet) package in R for variable selection

Sorry if this question comes across a little basic. I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
1
vote
0answers
18 views

Feature boosting via rescaling in logistic regression and linear SVMs

If I were expressing a problem in terms of binary features, all encoded as {0,1}, could I boost some features by encoding them as {0,2}? Would the effect change based on whether I used either of the ...
3
votes
0answers
82 views

Mutual information/pointwise mutual information for measuring prediction

I want to measure how well I predict a vector $Y$ (vector not a label) for observation $X$. Both $X$ and $Y$ have the same set of features ($1\times n$). For that, I thought of "scoring" the ...
0
votes
0answers
19 views

Statistical test for feature selection

Suppose I have a procedure to select K features out of M. I repeat the procedure N times on, say bootstrapped datasets, and count how many times feature #1 appeared among selected, denote k1, how many ...
0
votes
0answers
10 views

Feature selection based on cost function

Suppose that we are searching for best features using an optimization algorithm for a classification model (MLP,SNM,Regression,etc...). We should set a cost ...
2
votes
4answers
98 views

Interpreting conflicting results from Random Forest & Logistic Regression?

I am using SKLearn and Statsmodel in python to build a RF and Logistic Regression, respectively. I have a feature that the RF indicates is important (feature importance of 0.202, closely behind #1 ...
0
votes
0answers
18 views

Select the best point pair in the 2D grid [duplicate]

Suppose in the 2D space we have an array of points, and each point has a weighting factor, which is a float value ranging from 0 to 1. Each point also has a coordinate in the 2D grid. The following ...