Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
1answer
67 views

Getting less number of features in weight vectors as were provided for SVM

I have trained a SVM with 18881 features and wanted to know the ranking of features. I tried the method given at http://stackoverflow.com/questions/7390173/svm-equations-from-e1071-r-package for it ...
0
votes
0answers
347 views

K Fold Cross Validation, Variable Selection in LDA

I'm currently working on a multi-class classification problem and I attempt to use lda for the same. I have 2 questions here. 1) Is it possible to perform k-fold ...
0
votes
1answer
458 views

Large data variable selection

I'm looking for some methods of variable selection on large datasets.The number of variables are around 30-40, but the number of observations is quite large (around 36000000) Any methods which I ...
0
votes
0answers
43 views

Contribution to the components of a Gaussian mixture by data features

My question is about modelling data with a GMM using EM. One can split the mean and variance of each component into parts as well when working with data with multiple features. My question is what ...
1
vote
1answer
25 views

How to find the cause of defect in a process

Suppose a product A undergoes a certain process. This product A is produced at a rate of 8000 per month and out of those in 75 cases defects are generated. In the data set, I have rows corresponding ...
2
votes
1answer
110 views

Difference of variable selection and importance estimation

Isn't variable importance estimation a necessary prerequisite for variable selection? Is there any use case where you want to select non-important variables for your model? So, why is variable ...
0
votes
0answers
85 views

Multple linear regression, adding one predictor with almost perfect fit make others irrelevant

I found something interesting while playing with some data and linear regression. I built a regression with various predictors, more or less correlated with the outcome. Then I added one predictor ...
0
votes
1answer
47 views

Feature selection for unknown parametric model

Suppose one has about 500 points of 50 dimensional data that one knows a priori is derived from a parametric model (perhaps with some outliers). Does using this knowledge help in feature selection? I ...
2
votes
2answers
108 views

Do I need to take out any predictors from multiple regression if I put in some principal components as additional predictors?

I have an assignment which involves one area-level dataset made of $366$ scale variables. I have to perform PCA, compare it with rates of an additional response variable $X$, and comment on its face ...
0
votes
1answer
282 views

How to interpret random forest importance numbers

I ran randomForest in R package using 7 predictors variables (x1 to x7). I repeated the test with 4 dependent variables (y1 to y4). The importance numbers (IncNodePurity) are plotted in following ...
1
vote
2answers
516 views

How to build a predictive model with a billion of sparse features?

I am making a model to learn a dataset which has a big feature number and sparse samples (I am planning to use logistic regression). The feature number can be as big as 1,000,000,000. It is sparse ...
3
votes
0answers
64 views

Development data set for feature engineering and data exploration

I dont hear this being talked about much: If you want to engineer features and visually explore the data, should you do this on a development set separate from the training and test set? If ...
0
votes
0answers
31 views

Machine Learning: Potential Reasons of Precision Change after New Features are Added

My baseline model uses 10 features $[f_1, f_2, \dotsb, f_{10}]$. Now I have two new features $f_{11}$ and $f_{12}$. New models that use either $[f_1, f_2, \dotsb, f_{10}, f_{11}]$ or $[f_1, f_2, \...
0
votes
1answer
278 views

Chi-squared Vs Mutual information

Is chi-squared feature selection better than Mutual information based feature selection mechanism?
0
votes
1answer
100 views

Effect of combining features on classification

I have 2 string features F1 and F2 based on which I am trying to perform classification. I have two choices, either to use the ...
0
votes
1answer
84 views

After Clustering, how can I evaluate which features had the biggest impact?

I've just performed unsupervised clustering (using DBSCAN) on a dataset for which I have no expert knowledge on. I'm interested in working out which features had the greatest impact on my clustering. ...
2
votes
2answers
112 views

Backward feature selection with CV model selection

I am thinking about doing the following to a data set with $N$ samples and $m$ features 1) Train using semi-supervised learning and cross validate on labeled data using LOO-CV to select the best ...
1
vote
1answer
85 views

Variable selection and validation dataset

According to Hastie & Tibshirani, we shouldn't use validation datasets to do variable selection; otherwise, we will overestimate the model fit. However, it seems quite often to select variables ...
0
votes
1answer
162 views

NLP tokenization for building feature vector

I am trying to match new product description with the existing ones. Product description looks like this: Panasonic DMC-FX07EB digital camera silver. These are steps to be performed: Tokenize ...
2
votes
1answer
437 views

How to handle missing data in a small $n$ large $k$ machine learning scenario?

I have a sample size $N=130$ and $1000$ variables. I am using machine learning techniques (SVM) for analysing the data. Some variables in the dataset have values that are so huge that they must be ...
2
votes
1answer
469 views

Regarding the different variable selection result between regression modeling and random forest

I build a prediction modeling using both regression and random forest. ...
1
vote
0answers
31 views

Classification with two different dataset

I am working on a cancer classification model.Task is ,I am initially given a data set of 500 people and 1000 features.These people are given some kind of treatment(say Treatment 1). Some people are ...
4
votes
1answer
151 views

Placement of earlier features in more complex features in CNN

I'm trying to understand convolutional neural networks better. I've been doing different tutorials, but there are some basics concerning how the hidden units represents features that I really would ...
1
vote
2answers
156 views

How to best to use Continuous value features with discreet values for logistic regression based binary classification problem

This is related to Minimisation algorithm for a mix of discreet and continuous parameters? I am trying out logistic regression to solve a binary classification problem. Though I am feature-scaling ...
5
votes
1answer
133 views

“…if the data is linearly separable”

I keep hearing this phrase as a precursor to many algorithms, but I am not sure how exactly one goes about finding out if the data is indeed, linearly separable. Of course, if the data has ...
1
vote
1answer
504 views

Grid search for SVM parameters; is this is really how it is done?

Suppose I use nested 10-fold cross-validation with SVM. So, the inner-most loop will go around 100 times. Now, suppose I use a gaussian radial basis kernel function, which needs the parameter sigma. ...
1
vote
0answers
301 views

Different variable importance results with stabsel and mboost

I'm using glmboost in the mboost package to fit a boosted regression using linear models as the base learner. There are 13200 observations and about 75 variables, ...
0
votes
0answers
29 views

Alghorithm for choosing the best set of words for twitter filtering

I'm using the twitter API to get a stream of tweets. You can't get all the tweets from the public API, it requires you to add some word filters. But you can add up to 400 words for filtering and if a ...
1
vote
2answers
147 views

How to do cross-validation when comparing different feature selection methods?

I am using SVM for a prediction task. My sample size is small, only N=140. Suppose I want to compare the prediction accuracy when using two different feature selection methods. Would it be better to: ...
4
votes
1answer
686 views

Relationship between Gini Importance and Prediction Performance (say AUC)?

I want to use the decrease in Gini impurity to rank features for my random forest classifier. I understand that the decrease in Gini impurity at one node is calculated as: $$ \Delta i(n) = i(n) - ...
3
votes
1answer
2k views

How to analyze elastic net fitted model coefficients

SOLVED: an elastic net model, as any other logistic regression model, will not generate more coefficients than input variables. Check Zach's answer to understand how from an (apparent) low number of ...
1
vote
1answer
183 views

Stratified sampling for creating test/training sets when there are continous and categorical variables to consider?

Assume a simple clinical study with N=200. Half of the participants are men and half of the participants are women. The hemoglobin of the participants ranges between 80 and 150. There's also several ...
1
vote
2answers
300 views

Can we learn 3d features using Autoencoder?

Typically, we use Autoencoder to learn 2d features on 2d images (e.g. pen-strokes of digit). For example, if I have 10000 3d 31x31x31 images (e.g. car images). I unroll each of the images, i.e. ...
1
vote
0answers
32 views

Regression to chose questions which better correlate with a 10 points likert like score

We have a survey with several questions with 5 likert scale points and we would like to compare the answers to those of another likert like question with 10 points. The approach we thought of is a ...
1
vote
0answers
206 views

Using t-test for feature selection after z-scoring data?

Suppose I have a high-dimensional dataset, and a binary classification problem. I want to use the two-sample t-test for feature selection. If the data has been normalized by z-scoring (so it has zero ...
2
votes
1answer
118 views

How to adjust data to remove influence of one or more features

For my first real data science project I would like to develop a model which better reflects review quality than "useful" votes. I am working with Yelp's latest Academic data set but this thinking ...
3
votes
1answer
3k views

Removing collinear variables for LDA/QDA in R

I'm new to R and I've been searching for a while for a function which can reduce the number of explanatory variables in my lda function (linear discriminant analysis). Basically, I've loaded the ...
3
votes
0answers
135 views

LASSO prediction model question

I am trying to create a prediction model with 33 predictors (brain metabolite levels in various regions) and 8 observations (cognitive test scores) with p>>n problem using LASSO in MATLAB (...
0
votes
0answers
335 views

Selecting a multiple linear regression model with categorical variables

I am trying to analyze the Berkeley Guidance Study to practice multiple regression models, which has 10 continuous variables, 1 categorical variable (with two categories) and the response variable. ...
0
votes
1answer
647 views

Bi-normal separation feature selection (BNS) in R

I'm doing binary classification on highly dimensional text data, with a biased class distribution. After reading this paper, i found out about BNS feature selection. Is there any package that ...
1
vote
1answer
289 views

Tool form Hierarchical clustering

I'm trying to perform a hierarchical Clustering Analysis in a dataset of 40 attributes and +70,000 records, which is mostly composed by categorical variables. I've used Matlab and RapidMiner to ...
0
votes
0answers
37 views

Top K variable that represent entire dataset

There are 100 variables in the dataset. Also, i have extracted some additional information about each variable viz Var1 is correlated (Pearson correlation) to Var21,Var25,Var34,Var45,Var55 ; Var2 is ...
1
vote
1answer
769 views

(Automated) feature selection in multiple regression with categorical variables

I need a general guide on what are the appropriate approaches to automated feature selection in multiple regression with categorical variables. In my case, I have several numeric and categorical ...
1
vote
0answers
180 views

Why normalized feature weights for linear regression are bad feature importance predictors

I am trying to interpret a linear regression model. I assumed using absolute value of feature weight coefficients as indicators of influence of input variable onto output variable. However, it seems ...
1
vote
1answer
93 views

Strategy for Analyzing Data

I have been learning about Machine Learning (via Udacity) and Statistics (via Coursera) the past few months and trying to figure out a good way to combine them for a general approach to explaining ...
0
votes
1answer
205 views

Feature selection when bagging trees/random forest

I want to get a better understanding of feature selection and how the number of features affect performance when bagging trees. I am using Matlab's treebagger and I ...
3
votes
1answer
3k views

Feature Selection: Information Gain VS Mutual Information

Setting: Multi-class classification problem with discrete nominal features. There are many references mentioning the use of IG(Information Gain) and ...
0
votes
1answer
6 views

Select the most confident variable that has two features

Suppose now I have a group of students, and for each student two measurements are given: one is the height of the student and the other is the weight of the student. Then my question is how I can ...
0
votes
1answer
947 views

Weka java API: Attribute Selection and Cross Validation

Is there a way to perform Attirbute selection(aka feature selection) (regardless of method) only for the training dataset before passing data for Cross Validation ? I currently think that the only ...
0
votes
1answer
45 views

Use fitted value from regression on subset of features as independent variable

I am working with a relatively large data set with 2K columns and many variables can be grouped together (a logistic regression). So I am thinking can I use fitted value from regression on subset of ...