Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
1answer
469 views

Grid search for SVM parameters; is this is really how it is done?

Suppose I use nested 10-fold cross-validation with SVM. So, the inner-most loop will go around 100 times. Now, suppose I use a gaussian radial basis kernel function, which needs the parameter sigma. ...
1
vote
0answers
264 views

Different variable importance results with stabsel and mboost

I'm using glmboost in the mboost package to fit a boosted regression using linear models as the base learner. There are 13200 observations and about 75 variables, ...
0
votes
0answers
29 views

Alghorithm for choosing the best set of words for twitter filtering

I'm using the twitter API to get a stream of tweets. You can't get all the tweets from the public API, it requires you to add some word filters. But you can add up to 400 words for filtering and if a ...
1
vote
2answers
145 views

How to do cross-validation when comparing different feature selection methods?

I am using SVM for a prediction task. My sample size is small, only N=140. Suppose I want to compare the prediction accuracy when using two different feature selection methods. Would it be better to: ...
4
votes
1answer
678 views

Relationship between Gini Importance and Prediction Performance (say AUC)?

I want to use the decrease in Gini impurity to rank features for my random forest classifier. I understand that the decrease in Gini impurity at one node is calculated as: $$ \Delta i(n) = i(n) - ...
3
votes
1answer
1k views

How to analyze elastic net fitted model coefficients

SOLVED: an elastic net model, as any other logistic regression model, will not generate more coefficients than input variables. Check Zach's answer to understand how from an (apparent) low number of ...
1
vote
1answer
182 views

Stratified sampling for creating test/training sets when there are continous and categorical variables to consider?

Assume a simple clinical study with N=200. Half of the participants are men and half of the participants are women. The hemoglobin of the participants ranges between 80 and 150. There's also several ...
1
vote
2answers
286 views

Can we learn 3d features using Autoencoder?

Typically, we use Autoencoder to learn 2d features on 2d images (e.g. pen-strokes of digit). For example, if I have 10000 3d 31x31x31 images (e.g. car images). I unroll each of the images, i.e. ...
1
vote
0answers
31 views

Regression to chose questions which better correlate with a 10 points likert like score

We have a survey with several questions with 5 likert scale points and we would like to compare the answers to those of another likert like question with 10 points. The approach we thought of is a ...
1
vote
0answers
196 views

Using t-test for feature selection after z-scoring data?

Suppose I have a high-dimensional dataset, and a binary classification problem. I want to use the two-sample t-test for feature selection. If the data has been normalized by z-scoring (so it has zero ...
2
votes
1answer
113 views

How to adjust data to remove influence of one or more features

For my first real data science project I would like to develop a model which better reflects review quality than "useful" votes. I am working with Yelp's latest Academic data set but this thinking ...
3
votes
1answer
3k views

Removing collinear variables for LDA/QDA in R

I'm new to R and I've been searching for a while for a function which can reduce the number of explanatory variables in my lda function (linear discriminant analysis). Basically, I've loaded the ...
3
votes
0answers
131 views

LASSO prediction model question

I am trying to create a prediction model with 33 predictors (brain metabolite levels in various regions) and 8 observations (cognitive test scores) with p>>n problem using LASSO in MATLAB (...
0
votes
0answers
331 views

Selecting a multiple linear regression model with categorical variables

I am trying to analyze the Berkeley Guidance Study to practice multiple regression models, which has 10 continuous variables, 1 categorical variable (with two categories) and the response variable. ...
0
votes
1answer
619 views

Bi-normal separation feature selection (BNS) in R

I'm doing binary classification on highly dimensional text data, with a biased class distribution. After reading this paper, i found out about BNS feature selection. Is there any package that ...
1
vote
1answer
282 views

Tool form Hierarchical clustering

I'm trying to perform a hierarchical Clustering Analysis in a dataset of 40 attributes and +70,000 records, which is mostly composed by categorical variables. I've used Matlab and RapidMiner to ...
0
votes
0answers
36 views

Top K variable that represent entire dataset

There are 100 variables in the dataset. Also, i have extracted some additional information about each variable viz Var1 is correlated (Pearson correlation) to Var21,Var25,Var34,Var45,Var55 ; Var2 is ...
1
vote
1answer
748 views

(Automated) feature selection in multiple regression with categorical variables

I need a general guide on what are the appropriate approaches to automated feature selection in multiple regression with categorical variables. In my case, I have several numeric and categorical ...
1
vote
0answers
174 views

Why normalized feature weights for linear regression are bad feature importance predictors

I am trying to interpret a linear regression model. I assumed using absolute value of feature weight coefficients as indicators of influence of input variable onto output variable. However, it seems ...
1
vote
1answer
93 views

Strategy for Analyzing Data

I have been learning about Machine Learning (via Udacity) and Statistics (via Coursera) the past few months and trying to figure out a good way to combine them for a general approach to explaining ...
0
votes
1answer
196 views

Feature selection when bagging trees/random forest

I want to get a better understanding of feature selection and how the number of features affect performance when bagging trees. I am using Matlab's treebagger and I ...
3
votes
2answers
3k views

Feature Selection: Information Gain VS Mutual Information

Setting: Multi-class classification problem with discrete nominal features. There are many references mentioning the use of IG(Information Gain) and ...
0
votes
1answer
6 views

Select the most confident variable that has two features

Suppose now I have a group of students, and for each student two measurements are given: one is the height of the student and the other is the weight of the student. Then my question is how I can ...
0
votes
1answer
911 views

Weka java API: Attribute Selection and Cross Validation

Is there a way to perform Attirbute selection(aka feature selection) (regardless of method) only for the training dataset before passing data for Cross Validation ? I currently think that the only ...
0
votes
1answer
45 views

Use fitted value from regression on subset of features as independent variable

I am working with a relatively large data set with 2K columns and many variables can be grouped together (a logistic regression). So I am thinking can I use fitted value from regression on subset of ...
1
vote
1answer
228 views

mtry tuning given by caret higher than the number of predictors

According to this discussion, it seems that the train function of the caret package returns a ...
1
vote
1answer
283 views

Feature Importance in each fold and repeat after repeated cross validation in caret

this is my first post on Cross Validated so I apologize in advance if I'm not yet familiar with any conventions regarding forum posts. Currently, I'm working on a feature selection task using elastic ...
3
votes
1answer
147 views

Feature selection and training on the same sample

Is feature selection and training on the same sample a bad idea? I want to emphasize that I am not going to use test set for feature selection. If I use the whole train set for feature selection and ...
4
votes
1answer
2k views

Using topic words generated by LDA to represent a document

I want to do document classification by representing each document as a set of features. I know that there are many ways: BOW, TFIDF, ... I want to use Latent Dirichlet Allocation (LDA) to extract ...
1
vote
0answers
59 views

Feature boosting via rescaling in logistic regression and linear SVMs

If I were expressing a problem in terms of binary features, all encoded as {0,1}, could I boost some features by encoding them as {0,2}? Would the effect change based on whether I used either of the ...
0
votes
0answers
44 views

Statistical test for feature selection

Suppose I have a procedure to select K features out of M. I repeat the procedure N times on, say bootstrapped datasets, and count how many times feature #1 appeared among selected, denote k1, how many ...
0
votes
0answers
18 views

Select the best point pair in the 2D grid [duplicate]

Suppose in the 2D space we have an array of points, and each point has a weighting factor, which is a float value ranging from 0 to 1. Each point also has a coordinate in the 2D grid. The following ...
3
votes
2answers
217 views

Why isn't the lasso selected variable even not significant?

I performed a lasso selection using lars::lars for a well normally distributed outcome using a pool of 86 predictors. Here is the plot of the output: The ...
0
votes
1answer
132 views

Handle large set of features using SVM

I have a biological dataset with 30.000 features (genes) and 1000 data points (cells). Basically I have two major classes of cells: 1 and 0 with a distribution of 90/10. Now I am trying to classify ...
1
vote
0answers
70 views

How to deal with categorical features.

Recently I am playing in the famous Big-Data website Kaggle. There is a Display Advertising Challenge. In this competition, you are giving a training file which include huge records. the records is ...
1
vote
0answers
40 views

Does within-group heterogeneity negatively impact random forest classification?

I have two rather conceptual questions about random forest classifiers. Before we get there, I quickly want to lay out the problem I am working on: I have large a large data set consisting of 300 ...
1
vote
0answers
78 views

Considering non-i.i.d. covariates in random forests

Random forests are theoretically funded on the assumption that the data are i.i.d. realizations from a multivariate random vector $(X_1, \ldots, X_p, Y)$. Does it make sense to use random forests (for ...
1
vote
0answers
78 views

categorical feature ranking

I would like to rank categorical features by the order or importance in a classification/regression setting. Input There are two features, which are survey questions: "how is your mood?": four ...
2
votes
4answers
370 views

Interpreting conflicting results from Random Forest & Logistic Regression?

I am using SKLearn and Statsmodel in python to build a RF and Logistic Regression, respectively. I have a feature that the RF indicates is important (feature importance of 0.202, closely behind #1 ...
0
votes
0answers
821 views

Find linear SVM feature weights using libsvm

I'm trying to use linear SVM to do some feature selection. I'm using libsvm, but I cannot figure out how to find feature weights. The model file created looks something like this: ...
0
votes
1answer
184 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
0
votes
1answer
115 views

Feature selection with a binary dependent variable

Given we have a binary dependent variable and 100s of features and ~50k observations, is there a generally accepted way to trim the features via some type of machine learning concept? I was trying a ...
2
votes
2answers
193 views

What interactions to include in my GLM model?

I realize this might be a too general question, so I'll describe what I'm doing right now first. I'm working for a virtual insurance company and I have this dataset. It has severity (meaning ...
0
votes
1answer
653 views

Caret: customizing feature selection, nested inside cross validation

Using caret, I want to train a SVM classifier and estimate its performance using repeated cross validation. My dataset has a very large number of predictors (300K) and I want to reduce this number ...
0
votes
2answers
3k views

How to normalize time series?

This is a general question on normalization of data so that all the variables are within the same range. Why do we normalize data in pattern classification? How to normalize time series which is ...
0
votes
2answers
127 views

Feature Selection using (low) MCC

I have approximately 1200 input parameters that I am trying to whittle down with the following rough process: 1) Fit rbf SVM with n = 1200 parameters and calculate Matthews Correlation ...
3
votes
1answer
42 views

Boruta score goes to minus infinity

I'm running the Boruta algorithm with a $179\times 36$ predictor matrix and a numerical response. Most of the variables have a score going to -Inf. Should I ...
1
vote
0answers
22 views

How to assess the importance of the features which come from intersection of features of the two models?

I have two models from two different data sets. Model 1 contain 50 features and model 2 contain 40 features. the intersection of features of model 1 and 2 is 10. so how can I assess the relative ...
0
votes
0answers
77 views

How many features can be used for classification?

Asked a similar question the other day without an answer Link. I think maybe the question there is too big. Here I want to ask a specific one: 2 Class labels (Binary classification labelled with ...
0
votes
1answer
30 views

Including or discarding a variable based on its frequency

I am trying to dummy code a variable sector which have 20 levels. I am trying to combine some of levels. However I got this situation, if I have a say 2000 observations, how do I decide on the minimum ...