Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

4
votes
1answer
710 views

How to handle changing input vector length with neural networks

I want to train a neural network with a sequence of character as an input vector. Learning examples have different length and for this reason I don't know how to represent them. Let's say I have two ...
2
votes
1answer
205 views

Proper variable selection: Use only training data or full data?

I'm going through the lab exercises in "Introduction to Statistical Learning" and am having difficulty understanding the proper way to do best subset selection. The book is available here (http://www-...
1
vote
0answers
66 views

Elastic net is being used in genome wide analysis. Similar approach would work for survey analysis?

I'm approaching the elastic net procedure for genome wide analysis (GWAS) because it allows for feature selection, groups detection and improved validity. It's a powerful technique when you have many ...
2
votes
0answers
37 views

Can I reuse the dataset set aside for performing t-test based on the following condition?

I have a small number of samples and large number features. For doing the feature selection I'm going to divide my total set into a feature selection set and a test set.I run the t-test on the former ...
4
votes
1answer
533 views

What is the difference between feature selection and dimensionality reduction?

I know that both feature selection and dimensionality reduction aim towards reducing the number of features in the original set of features. What is the exact difference between the two if we are ...
2
votes
1answer
147 views

SVM Classifier: proper process

I'm working on a classification system of mine, but am needing help with the proper process order. Specifically, I'm using LibSVM and a range of feature sets extracted from my data. I'm wondering, ...
0
votes
1answer
284 views

determining how “important” a feature is in predicting a target in decision trees

Random forests allow us to compute a heuristic for determining how "important" a feature is in predicting a target. This heuristic measures the change in prediction accuracy if we take a given ...
1
vote
1answer
66 views

Do Neural Networks need “compound” features?

Apologies if I haven't got the terminology quite right. I have a question about Neural Networks, and I'm not sure exactly the best way to ask it! Hypothetically, let's say I have a dataset of houses ...
1
vote
1answer
790 views

What does the varImp function in the caret package actually compute for a glmnet (elastic net) object

I am fitting an elastic net model with glmnet via the caret package with 189 predictors and a binomial criteria (a,b) ...
0
votes
0answers
42 views

What is the best practices as to cropping the positive examples for training hog classifier?

I've been reading up on the literature on HOG training, but I can't figure out what the best practices are for cropping the positive examples. Question: Do you crop tight or with margins on the “...
2
votes
1answer
198 views

K-means clustering feature selection

I have a set of English and foreign language documents that I would to perform k-means clustering on to find document groups by topic. These documents are concatenated social media comments for ...
1
vote
0answers
90 views

Poisson model with fractions

I have a simple website with a home page that has 5 different images on it. All images have a fixed set of 'features' associated with them (size, color, position etc.,). When a visitor comes to the ...
0
votes
1answer
270 views

Selecting variables in multiple linear regression in R

Consider that we have a problem with 4 variables (y, x1, x2 and x3) and we want to do a multiple linear regression model. As we need to know which variables are the most important in the problem, we ...
1
vote
1answer
31 views

Multi Categorical Features vs multiple Features for categories

Say I am discretizing continuous data based on percentiles. (I realize this is generally frowned upon, but I am doing this for the sake of experiment) I am trying different percentiles, eg breaking ...
0
votes
1answer
194 views

combining multiple classifiers common features

Can multiple binary-classifiers be combined to produce a final output if their feature sets have some common elements? How will this influence the accuracy?
0
votes
1answer
40 views

Using a set as a feature in decision tree classification

I'm faced with a data set where one of the features is a set of 4-5 categories (this number of categories isn't constant). I need to use this feature for building a decision tree. I searched online ...
3
votes
1answer
144 views

Comparing classification results from models selected differently with Kruskal-Wallis

For a seminar at university I'm reviewing a paper that proposes a new feature selection algorithm. The authors also evaluated their algorithm by applying it beside two other feature selection ...
1
vote
1answer
69 views

Getting less number of features in weight vectors as were provided for SVM

I have trained a SVM with 18881 features and wanted to know the ranking of features. I tried the method given at http://stackoverflow.com/questions/7390173/svm-equations-from-e1071-r-package for it ...
0
votes
0answers
369 views

K Fold Cross Validation, Variable Selection in LDA

I'm currently working on a multi-class classification problem and I attempt to use lda for the same. I have 2 questions here. 1) Is it possible to perform k-fold ...
0
votes
1answer
486 views

Large data variable selection

I'm looking for some methods of variable selection on large datasets.The number of variables are around 30-40, but the number of observations is quite large (around 36000000) Any methods which I ...
0
votes
0answers
43 views

Contribution to the components of a Gaussian mixture by data features

My question is about modelling data with a GMM using EM. One can split the mean and variance of each component into parts as well when working with data with multiple features. My question is what ...
1
vote
1answer
25 views

How to find the cause of defect in a process

Suppose a product A undergoes a certain process. This product A is produced at a rate of 8000 per month and out of those in 75 cases defects are generated. In the data set, I have rows corresponding ...
2
votes
1answer
112 views

Difference of variable selection and importance estimation

Isn't variable importance estimation a necessary prerequisite for variable selection? Is there any use case where you want to select non-important variables for your model? So, why is variable ...
0
votes
0answers
87 views

Multple linear regression, adding one predictor with almost perfect fit make others irrelevant

I found something interesting while playing with some data and linear regression. I built a regression with various predictors, more or less correlated with the outcome. Then I added one predictor ...
0
votes
1answer
48 views

Feature selection for unknown parametric model

Suppose one has about 500 points of 50 dimensional data that one knows a priori is derived from a parametric model (perhaps with some outliers). Does using this knowledge help in feature selection? I ...
2
votes
2answers
112 views

Do I need to take out any predictors from multiple regression if I put in some principal components as additional predictors?

I have an assignment which involves one area-level dataset made of $366$ scale variables. I have to perform PCA, compare it with rates of an additional response variable $X$, and comment on its face ...
0
votes
1answer
292 views

How to interpret random forest importance numbers

I ran randomForest in R package using 7 predictors variables (x1 to x7). I repeated the test with 4 dependent variables (y1 to y4). The importance numbers (IncNodePurity) are plotted in following ...
1
vote
2answers
553 views

How to build a predictive model with a billion of sparse features?

I am making a model to learn a dataset which has a big feature number and sparse samples (I am planning to use logistic regression). The feature number can be as big as 1,000,000,000. It is sparse ...
3
votes
0answers
64 views

Development data set for feature engineering and data exploration

I dont hear this being talked about much: If you want to engineer features and visually explore the data, should you do this on a development set separate from the training and test set? If ...
0
votes
0answers
31 views

Machine Learning: Potential Reasons of Precision Change after New Features are Added

My baseline model uses 10 features $[f_1, f_2, \dotsb, f_{10}]$. Now I have two new features $f_{11}$ and $f_{12}$. New models that use either $[f_1, f_2, \dotsb, f_{10}, f_{11}]$ or $[f_1, f_2, \...
0
votes
1answer
303 views

Chi-squared Vs Mutual information

Is chi-squared feature selection better than Mutual information based feature selection mechanism?
0
votes
1answer
105 views

Effect of combining features on classification

I have 2 string features F1 and F2 based on which I am trying to perform classification. I have two choices, either to use the ...
0
votes
1answer
86 views

After Clustering, how can I evaluate which features had the biggest impact?

I've just performed unsupervised clustering (using DBSCAN) on a dataset for which I have no expert knowledge on. I'm interested in working out which features had the greatest impact on my clustering. ...
2
votes
2answers
113 views

Backward feature selection with CV model selection

I am thinking about doing the following to a data set with $N$ samples and $m$ features 1) Train using semi-supervised learning and cross validate on labeled data using LOO-CV to select the best ...
1
vote
1answer
89 views

Variable selection and validation dataset

According to Hastie & Tibshirani, we shouldn't use validation datasets to do variable selection; otherwise, we will overestimate the model fit. However, it seems quite often to select variables ...
0
votes
1answer
166 views

NLP tokenization for building feature vector

I am trying to match new product description with the existing ones. Product description looks like this: Panasonic DMC-FX07EB digital camera silver. These are steps to be performed: Tokenize ...
2
votes
1answer
450 views

How to handle missing data in a small $n$ large $k$ machine learning scenario?

I have a sample size $N=130$ and $1000$ variables. I am using machine learning techniques (SVM) for analysing the data. Some variables in the dataset have values that are so huge that they must be ...
2
votes
1answer
505 views

Regarding the different variable selection result between regression modeling and random forest

I build a prediction modeling using both regression and random forest. ...
1
vote
0answers
31 views

Classification with two different dataset

I am working on a cancer classification model.Task is ,I am initially given a data set of 500 people and 1000 features.These people are given some kind of treatment(say Treatment 1). Some people are ...
4
votes
1answer
155 views

Placement of earlier features in more complex features in CNN

I'm trying to understand convolutional neural networks better. I've been doing different tutorials, but there are some basics concerning how the hidden units represents features that I really would ...
1
vote
2answers
164 views

How to best to use Continuous value features with discreet values for logistic regression based binary classification problem

This is related to Minimisation algorithm for a mix of discreet and continuous parameters? I am trying out logistic regression to solve a binary classification problem. Though I am feature-scaling ...
5
votes
1answer
133 views

“…if the data is linearly separable”

I keep hearing this phrase as a precursor to many algorithms, but I am not sure how exactly one goes about finding out if the data is indeed, linearly separable. Of course, if the data has ...
1
vote
1answer
527 views

Grid search for SVM parameters; is this is really how it is done?

Suppose I use nested 10-fold cross-validation with SVM. So, the inner-most loop will go around 100 times. Now, suppose I use a gaussian radial basis kernel function, which needs the parameter sigma. ...
1
vote
0answers
316 views

Different variable importance results with stabsel and mboost

I'm using glmboost in the mboost package to fit a boosted regression using linear models as the base learner. There are 13200 observations and about 75 variables, ...
0
votes
0answers
29 views

Alghorithm for choosing the best set of words for twitter filtering

I'm using the twitter API to get a stream of tweets. You can't get all the tweets from the public API, it requires you to add some word filters. But you can add up to 400 words for filtering and if a ...
1
vote
2answers
148 views

How to do cross-validation when comparing different feature selection methods?

I am using SVM for a prediction task. My sample size is small, only N=140. Suppose I want to compare the prediction accuracy when using two different feature selection methods. Would it be better to: ...
4
votes
1answer
698 views

Relationship between Gini Importance and Prediction Performance (say AUC)?

I want to use the decrease in Gini impurity to rank features for my random forest classifier. I understand that the decrease in Gini impurity at one node is calculated as: $$ \Delta i(n) = i(n) - ...
3
votes
1answer
2k views

How to analyze elastic net fitted model coefficients

SOLVED: an elastic net model, as any other logistic regression model, will not generate more coefficients than input variables. Check Zach's answer to understand how from an (apparent) low number of ...
1
vote
1answer
190 views

Stratified sampling for creating test/training sets when there are continous and categorical variables to consider?

Assume a simple clinical study with N=200. Half of the participants are men and half of the participants are women. The hemoglobin of the participants ranges between 80 and 150. There's also several ...
1
vote
2answers
316 views

Can we learn 3d features using Autoencoder?

Typically, we use Autoencoder to learn 2d features on 2d images (e.g. pen-strokes of digit). For example, if I have 10000 3d 31x31x31 images (e.g. car images). I unroll each of the images, i.e. ...