Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
1answer
217 views

mtry tuning given by caret higher than the number of predictors

According to this discussion, it seems that the train function of the caret package returns a ...
0
votes
1answer
273 views

Feature Importance in each fold and repeat after repeated cross validation in caret

this is my first post on Cross Validated so I apologize in advance if I'm not yet familiar with any conventions regarding forum posts. Currently, I'm working on a feature selection task using elastic ...
3
votes
1answer
145 views

Feature selection and training on the same sample

Is feature selection and training on the same sample a bad idea? I want to emphasize that I am not going to use test set for feature selection. If I use the whole train set for feature selection and ...
3
votes
1answer
2k views

Using topic words generated by LDA to represent a document

I want to do document classification by representing each document as a set of features. I know that there are many ways: BOW, TFIDF, ... I want to use Latent Dirichlet Allocation (LDA) to extract ...
1
vote
0answers
57 views

Feature boosting via rescaling in logistic regression and linear SVMs

If I were expressing a problem in terms of binary features, all encoded as {0,1}, could I boost some features by encoding them as {0,2}? Would the effect change based on whether I used either of the ...
0
votes
0answers
41 views

Statistical test for feature selection

Suppose I have a procedure to select K features out of M. I repeat the procedure N times on, say bootstrapped datasets, and count how many times feature #1 appeared among selected, denote k1, how many ...
0
votes
0answers
18 views

Select the best point pair in the 2D grid [duplicate]

Suppose in the 2D space we have an array of points, and each point has a weighting factor, which is a float value ranging from 0 to 1. Each point also has a coordinate in the 2D grid. The following ...
3
votes
2answers
204 views

Why isn't the lasso selected variable even not significant?

I performed a lasso selection using lars::lars for a well normally distributed outcome using a pool of 86 predictors. Here is the plot of the output: The ...
0
votes
1answer
127 views

Handle large set of features using SVM

I have a biological dataset with 30.000 features (genes) and 1000 data points (cells). Basically I have two major classes of cells: 1 and 0 with a distribution of 90/10. Now I am trying to classify ...
1
vote
0answers
67 views

How to deal with categorical features.

Recently I am playing in the famous Big-Data website Kaggle. There is a Display Advertising Challenge. In this competition, you are giving a training file which include huge records. the records is ...
1
vote
0answers
39 views

Does within-group heterogeneity negatively impact random forest classification?

I have two rather conceptual questions about random forest classifiers. Before we get there, I quickly want to lay out the problem I am working on: I have large a large data set consisting of 300 ...
1
vote
0answers
78 views

Considering non-i.i.d. covariates in random forests

Random forests are theoretically funded on the assumption that the data are i.i.d. realizations from a multivariate random vector $(X_1, \ldots, X_p, Y)$. Does it make sense to use random forests (for ...
1
vote
0answers
77 views

categorical feature ranking

I would like to rank categorical features by the order or importance in a classification/regression setting. Input There are two features, which are survey questions: "how is your mood?": four ...
2
votes
4answers
358 views

Interpreting conflicting results from Random Forest & Logistic Regression?

I am using SKLearn and Statsmodel in python to build a RF and Logistic Regression, respectively. I have a feature that the RF indicates is important (feature importance of 0.202, closely behind #1 ...
0
votes
0answers
797 views

Find linear SVM feature weights using libsvm

I'm trying to use linear SVM to do some feature selection. I'm using libsvm, but I cannot figure out how to find feature weights. The model file created looks something like this: ...
0
votes
1answer
169 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
0
votes
1answer
110 views

Feature selection with a binary dependent variable

Given we have a binary dependent variable and 100s of features and ~50k observations, is there a generally accepted way to trim the features via some type of machine learning concept? I was trying a ...
2
votes
2answers
183 views

What interactions to include in my GLM model?

I realize this might be a too general question, so I'll describe what I'm doing right now first. I'm working for a virtual insurance company and I have this dataset. It has severity (meaning ...
0
votes
1answer
615 views

Caret: customizing feature selection, nested inside cross validation

Using caret, I want to train a SVM classifier and estimate its performance using repeated cross validation. My dataset has a very large number of predictors (300K) and I want to reduce this number ...
0
votes
2answers
3k views

How to normalize time series?

This is a general question on normalization of data so that all the variables are within the same range. Why do we normalize data in pattern classification? How to normalize time series which is ...
0
votes
2answers
125 views

Feature Selection using (low) MCC

I have approximately 1200 input parameters that I am trying to whittle down with the following rough process: 1) Fit rbf SVM with n = 1200 parameters and calculate Matthews Correlation ...
3
votes
1answer
42 views

Boruta score goes to minus infinity

I'm running the Boruta algorithm with a $179\times 36$ predictor matrix and a numerical response. Most of the variables have a score going to -Inf. Should I ...
1
vote
0answers
22 views

How to assess the importance of the features which come from intersection of features of the two models?

I have two models from two different data sets. Model 1 contain 50 features and model 2 contain 40 features. the intersection of features of model 1 and 2 is 10. so how can I assess the relative ...
0
votes
0answers
74 views

How many features can be used for classification?

Asked a similar question the other day without an answer Link. I think maybe the question there is too big. Here I want to ask a specific one: 2 Class labels (Binary classification labelled with ...
0
votes
1answer
29 views

Including or discarding a variable based on its frequency

I am trying to dummy code a variable sector which have 20 levels. I am trying to combine some of levels. However I got this situation, if I have a say 2000 observations, how do I decide on the minimum ...
6
votes
6answers
2k views

Methods in R or Python to perform feature selection in unsupervised learning

What are the available methods/implementation in R/Python to discard/select unimportant/important features in data? My data does not have labels (unsupervised). The data has ~100 features with mixed ...
1
vote
1answer
151 views

Feature Selection - Mutual Information with response variable that takes three values

I am trying to calculate Mutual Information scores for Feature Selection. I have successfully implemented the Mutual Information to test each feature against the binary response variable. Each ...
1
vote
1answer
728 views

Why does SAS Enterprise Miner keep all dummy variables for a coded categorical variable in stepwise logistic regression?

SAS Enterprise Miner nicely creates coded dummy variables for any categorical variables used in a logistic regression model. When it performs a variable selection using stepwise sequential selection ...
1
vote
0answers
75 views

Using Leave-One-Out Cross Validation with LARS

I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such: $$\mathbb{min}\ ...
1
vote
0answers
26 views

Setting up feature vectors

I am working on a classification project and I want to use SVM's and/or Clustering Algs. What I am having trouble with is deciding how to set up my feature vectors. I have already decided what my ...
2
votes
0answers
93 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
1
vote
1answer
223 views

Maximum Entropy Model for classification, what to use as context & feature?

I'm building a Maximum Entropy Model to classify some text, based on paper "A Maximum Entropy Approach to Natural Language Processing" by Berger et.al. It's similar to POS tagging. Below is some ...
3
votes
2answers
2k views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
1
vote
1answer
4k views

Feature selection : how to select the Information Gain threshold?

I am trying to use Information Gain to select features when classifying text with a Support Vector Machine. For each word in our training data, we computed its information gain. Then, we should keep ...
0
votes
1answer
295 views

Feature Normalization/Standardization before or after Feature Selection?

Should the process of feature normalization/standardization be done before or after the feature selection process?
2
votes
2answers
598 views

Model Tuning and Model Evaluation in Machine Learning

Despite my readings (on stack 1, 2, or in literature (Cawley, 2010; Japkowicz, 2011)), I don't find a clear procedure for tuning and evaluating a model in a classification task. I want to perform a ...
1
vote
0answers
165 views

Unstable models, repeated crossvalidation, feature selection

I'm still trying to classify few (about 200) samples in a high dimensional feature space (dim=19) into 3 (very unbalanced) classes. I use an implementation of Least Squares SVM with one vs one coding ...
2
votes
2answers
4k views

Best feature selection method for naive Bayes classification

i want to make classification with naive Bayes. I have got about 100 Features. Numerical ones as well as categorical ones. Since i want only the most relevant ones to be included for the ...
1
vote
0answers
36 views

Removing insignificant variables? [duplicate]

Suppose you fit a linear regression model on some data with 10 variables. The F-statistic shows that 3 of them are significant (p < 0.05) , 2 are within trend (0.05 < p < 0.10), and the other ...
1
vote
0answers
126 views

Is this normal to have big value of Chi Squared?

I am using chi squared for feature selection in text classification. However when I compute it I sometime have very big values. Like 100, 1000 or even 20000. Is this normal ? I wonder because I ...
3
votes
2answers
2k views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
1
vote
0answers
294 views

How to apply feature selection based on tf-idf threshold

Let's say we have the following matrix (typical VSM example): ...
0
votes
1answer
438 views

Find variables selected for each subset using caret feature selection

I am doing feature selection using the command 'rfe' in the caret package (http://caret.r-forge.r-project.org/featureselection.html). This command uses a metric to find the optimal amount of variables ...
3
votes
2answers
933 views

Feature / attribute selection for k-means or other clustering

It seems to me that in literature it is assumed that one knows which features / attributes to choose to characterize an item in clustering. If I have a database with items which have many attributes, ...
0
votes
1answer
45 views

Importance of Time Features

if you have a time series and you want to do some predictions, what time feature should you use ? lets say we are trying to predict how many people visit a certain website, we have data for the ...
4
votes
2answers
319 views

Feature selection with partial permutation

For feature selection, permutation tests are biased in favor of those categorical variables with a large number of levels [White1994]. Besides, it has been proposed [Deng2011] that partial ...
0
votes
0answers
101 views

Regression by multiple dependent variables with constraints & feature selection

I have a data set of 1000 records. Each record has three dependent variables $y_1, y_2, y_3$ and 100 independent variables $x_1,...,x_{100}$, where the dependent variable $y_i$ satisfies: $0\le y_i ...
2
votes
2answers
999 views

How to select a subset of variables from my original long list in order to perform logistic regression analysis?

My situation: small sample size: 116 binary outcome variable long list of explanatory variables: 44 explanatory variables did not come from the top of my head; their choice was based on the ...
3
votes
2answers
181 views

Why is feature scaling or weighting important in supervised learning?

I can understand that feature scaling or weighting is important in unsupervised learning, because we want a good representation of "similarity". But why is it also important in the supervised learning ...
4
votes
1answer
124 views

Feature selection: permutation test Vs deleting a variable

In feature selection for predictive models, it is usually applied a permutation test. In this test, all the values of one variable are randomly permuted and the prediction accuracy is extracted for ...