Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
7 views

How to assess the importance of the features which come from intersection of features of the two models?

I have two models from two different data sets. Model 1 contain 50 features and model 2 contain 40 features. the intersection of features of model 1 and 2 is 10. so how can I assess the relative ...
0
votes
0answers
12 views

How many features can be used for classification?

Asked a similar question the other day without an answer Link. I think maybe the question there is too big. Here I want to ask a specific one: 2 Class labels (Binary classification labelled with ...
7
votes
3answers
942 views

Computing best subset of predictors for linear regression

For the selection of predictors in multivariate linear regression with $p$ suitable predictors, what methods are available to find an 'optimal' subset of the predictors without explicitly testing all ...
0
votes
1answer
22 views

Including or discarding a variable based on its frequency

I am trying to dummy code a variable sector which have 20 levels. I am trying to combine some of levels. However I got this situation, if I have a say 2000 observations, how do I decide on the minimum ...
1
vote
0answers
33 views

methods to perform feature selection in unsupervissed learning

What are the available methods/implementation in R/Python to discard/select unimportant/important features in data? My data does not have labels (unsupervised). The data has ~100 features with mixed ...
1
vote
1answer
72 views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
0
votes
1answer
292 views

What is a good Gini decrease cutoff for feature inclusion based upon random forests?

I am using random forests to try and determine variable importance as part of feature selection for a model I'm working on, and while I can get ranked variable importance by mean decrease in Gini from ...
1
vote
1answer
20 views

Feature Selection - Mutual Information with response variable that takes three values

I am trying to calculate Mutual Information scores for Feature Selection. I have successfully implemented the Mutual Information to test each feature against the binary response variable. Each ...
2
votes
1answer
143 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
1
vote
1answer
195 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
0
votes
0answers
24 views

How many features to overfit the classification?

Recently, I have got some 'strange' comments from the reviewer of my paper. In my paper, I discussed a novel feature extraction method, and then I compared three classification methods for my binary ...
1
vote
1answer
25 views

Why does SAS Enterprise Miner keep all dummy variables for a coded categorical variable in stepwise logistic regression?

SAS Enterprise Miner nicely creates coded dummy variables for any categorical variables used in a logistic regression model. When it performs a variable selection using stepwise sequential selection ...
3
votes
1answer
415 views
1
vote
0answers
29 views

Using Leave-One-Out Cross Validation with LARS

I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such: $$\mathbb{min}\ ...
-1
votes
1answer
185 views

Implement Forward, Backward, Step and LASSO in VB .NET

My client wants me to implement Variable selection methods i.e. Forward, Backward, Step and LASSO in VB .Net platform including p-value and AIC. I have no idea about the steps involved to calculate ...
1
vote
0answers
20 views

Setting up feature vectors

I am working on a classification project and I want to use SVM's and/or Clustering Algs. What I am having trouble with is deciding how to set up my feature vectors. I have already decided what my ...
1
vote
2answers
129 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
0
votes
0answers
16 views

Is subtracting the mean from PCA necessary when using an SVD result that is feature scaled?

I've applied SVD to the original data matrix and eliminated insignificant columns and rows from U and V^T respectively using the Sigma values. I multiplied together my optimized U, Sigma, and V^T ...
2
votes
1answer
74 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
3
votes
2answers
66 views

Advice for interpolating a model

I'm new in Stack Exchange, so I hope no to be off topic. I'm also new in bioinformatics and I was asked to perform an analysis. Briefly, I have a dataset of 29 cell lines and the IC50 values of a test ...
0
votes
0answers
9 views

Feature selection: all features vs a subset of them

I am doing a binary classification. The dataset has 3000 samples, and each sample has 10 features. But I find that the performance of using all 10 features is almost the same as that of using only the ...
0
votes
1answer
90 views

How to find important features in this problem

I was just thinking how ML techniques can be applied in the retail industry. Suppose we have data from a retailer who deals with apparel and cloth in this format and for each item there are ...
0
votes
0answers
9 views

How to interpret merits in Weka with ChiSquaredAttributeEval and SVMAttributeEval?

I want to interpret the goodness of attributes using feature selection with 10-fold cross validation. With ChiSquared I get something like this (deletet attributes with merrit was 0 in all folds): ...
2
votes
0answers
33 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
0
votes
0answers
14 views

Feature selection from wavelet transformation in R

I am new to wavelets. Currently, I am developing a prediction model using time series data. I am using the wavelets package in R. I am taking part of the time ...
1
vote
1answer
56 views

Maximum Entropy Model for classification, what to use as context & feature?

I'm building a Maximum Entropy Model to classify some text, based on paper "A Maximum Entropy Approach to Natural Language Processing" by Berger et.al. It's similar to POS tagging. Below is some ...
8
votes
2answers
2k views

Finding the best features in interaction models

I have list of proteins with their feature values. A sample table looks like this: ...
1
vote
1answer
25 views

Significance Test for Comparison of Variables

I am not sure how to ask this question without giving an example. I am trying to measure the "cleanliness" of office buildings. I have two variables that try to measure this. Variable one is a ...
0
votes
1answer
79 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
0
votes
0answers
21 views

Regression by multiple dependent variables with constraints & feature selection

I have a data set of 1000 records. Each record has three dependent variables $y_1, y_2, y_3$ and 100 independent variables $x_1,...,x_{100}$, where the dependent variable $y_i$ satisfies: $0\le y_i ...
0
votes
0answers
8 views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
1
vote
1answer
50 views

Feature selection : how to select the Information Gain threshold?

I am trying to use Information Gain to select features when classifying text with a Support Vector Machine. For each word in our training data, we computed its information gain. Then, we should keep ...
0
votes
1answer
31 views

Feature Normalization/Standardization before or after Feature Selection?

Should the process of feature normalization/standardization be done before or after the feature selection process?
2
votes
2answers
80 views

Model Tuning and Model Evaluation in Machine Learning

Despite my readings (on stack 1, 2, or in literature (Cawley, 2010; Japkowicz, 2011)), I don't find a clear procedure for tuning and evaluating a model in a classification task. I want to perform a ...
0
votes
0answers
21 views

recursive feature elemination in R with caret

i work with R caret software package to select the most important features from some set of data. My response is a factor of multiple classes (e.g. nominal ...
0
votes
0answers
6 views

Nonlinear functions of other features as new features in SVM model with RBF kernel

Can some one give me some conceptual insight on the potential advantages of disadvantages of adding features that are (nonlinear) functions of existing features in training an SVM model with an RBF ...
0
votes
2answers
66 views

Choosing the best featureset for prediction

I have this N sets of features F each with $F_i$ number of features. All the feature sets have 20000 examples and we have 20,000 labels for them. Lets say feature set $F_1$ has 10 features and ...
0
votes
1answer
68 views

What is parameter fine tuning means in SVM?

I got this sentence in one of paper, but I dont understand what does it mean?? "Training a learningbased classifier such as an SVM on an imbalanced dataset often requires parameter fine-tuning, ...
1
vote
1answer
51 views

finding an optimal subgroup of binary indicators

My dependent variable is continuous variable that measures the (potential) success of a person in some activity. I have hundreds of binary indicators, each indicates about the existence of a specific ...
1
vote
0answers
23 views

Unstable models, repeated crossvalidation, feature selection

I'm still trying to classify few (about 200) samples in a high dimensional feature space (dim=19) into 3 (very unbalanced) classes. I use an implementation of Least Squares SVM with one vs one coding ...
0
votes
2answers
43 views

Best feature selection method for naive Bayes classification

i want to make classification with naive Bayes. I have got about 100 Features. Numerical ones as well as categorical ones. Since i want only the most relevant ones to be included for the ...
0
votes
0answers
18 views

TF-IDF for text classification by taking into account the document class

I am looking for a TF-IDF weighting for text classification (not document ranking/retrieval) which takes also into account the document class. For example let's use the typical spam/not spam ...
4
votes
4answers
5k views

Using LASSO from lars (or glmnet) package in R for variable selection

Sorry if this question comes across a little basic. I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
1
vote
0answers
64 views

Is this normal to have big value of Chi Squared?

I am using chi squared for feature selection in text classification. However when I compute it I sometime have very big values. Like 100, 1000 or even 20000. Is this normal ? I wonder because I ...
1
vote
0answers
27 views

Removing insignificant variables? [duplicate]

Suppose you fit a linear regression model on some data with 10 variables. The F-statistic shows that 3 of them are significant (p < 0.05) , 2 are within trend (0.05 < p < 0.10), and the other ...
2
votes
2answers
281 views

How to select a subset of variables from my original long list in order to perform logistic regression analysis?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
1
vote
1answer
60 views

Feature selection for pattern mining

I must find frequent patterns in temporal data, using a method that was imposed to me. This tool has problems handling these data: processing is long and takes a lot of memory. So, I would like to ...
0
votes
1answer
22 views

Find variables selected for each subset using caret feature selection

I am doing feature selection using the command 'rfe' in the caret package (http://caret.r-forge.r-project.org/featureselection.html). This command uses a metric to find the optimal amount of variables ...
5
votes
3answers
987 views

How to avoid overfitting when using crossvalidation within Genetic Algorithms

This is a long set-up, but the pure intellectual challenge will make it worthwhile I promise ;-) I have marketing data where there is a treatment and a control (i.e a customer gets no treatment). The ...