Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

3
votes
1answer
795 views

Justification for feature selection by removing predictors with near zero variance

I have a large number of variables that I'm trying to reduce, and I've stumbled on Kuhn's (2008) suggestion to eliminate variables with zero or near-zero variance: [Near-zero variance means that ...
2
votes
2answers
398 views

Random Forests for predictor importance (Matlab)

I'm working with a dataset of approx 150,000 observations and 50 features, using SVM for the final model. To trim down the feature count, I decided to look into using RF so SVM optimization doesn't ...
1
vote
0answers
18 views

Why do loadings of princomp in R report identical proportion of variance for all principal components? [duplicate]

I'm trying to run a few tests using princomp in R. In princomp there is a value called <...
3
votes
1answer
558 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
0
votes
1answer
9 views

Correlation based feature selection(CFS) tool

Is there any tool or script that was implemented for correlation based feature selection? My feature vector data is in a large-scaled data file, so if I use tools like Weka for feature selection, I ...
0
votes
1answer
20 views

Consequences of overlap between training, validation and test data

If I'm splitting my data in training, validation and test data to assess different (sub)sets of features for my task. What are the consequences if I (by mistake) split my data incorrectly? In the ...
0
votes
0answers
4 views

how to define the Features,labels and function in sequentialfs matlab [on hold]

I have the features extracted using LBP for 5 classes of size 645 X 40960 and labels of size 1X645. Now i want to select the relevant feature using sequentialfs..by going through matlab help..i tried ...
4
votes
4answers
112 views

Can PCA allow to identify redundant variables that can be removed before doing cluster analysis?

I hope this is suitable for this forum: I am new to PCA and what I ultimately want to do is perform cluster analysis on my dataset. I have 20 physical descriptor variables for organisms, each with ...
0
votes
2answers
131 views

Feature selection clustering customer segmentation

based on customer data I want to perform a clustering using different clustering algorithms (K-Means, Expectation Maximization, etc.) in R. The most attributes were engineered pursuing the goal to be ...
0
votes
0answers
19 views

Correlation for feature selection in multi-dimensional time series

I have multi-dimensional time series data with seven dimensions. The correlation coefficient between two of these dimensions is about 0.65. Can those two variables be said to be/treated like being ...
26
votes
2answers
3k views

Why does the Lasso provide Variable Selection?

I've been reading Elements of Statistical Learning, and I would like to know why the Lasso provides variable selection and ridge regression doesn't. Both methods minimize the residual sum of squares ...
3
votes
1answer
696 views

Linear regression for feature selection

Imagine we regress y on x1...x4. Now, we want to find out if ...
1
vote
0answers
32 views

How the Correlation Matrix is built for PCA in Weka?

Just to give a context, I want to use PCA (Principal Component Analysis) to identify which attributes are similar to others, so I can use just one (or a subset) of them. The correlation matrix of n ...
0
votes
1answer
57 views

Feature/variable selection with categorical variables [on hold]

My goal is to compare several machine learning algorithms for sales prediction (logistic regression, neural network, random forest, svm -> classification problem, whether the sales will go up or down) ...
0
votes
0answers
3 views

best debug procedure for caret variable selection 'functions'? [migrated]

I'm trying to use caret's 'filter' functions to do variable selections before training a model using the following code: ...
2
votes
2answers
458 views

LASSO vs AIC for feature selection with the Cox model

I have some questions about the Lasso. After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, p-...
9
votes
1answer
2k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
2
votes
3answers
2k views

The disadvantage of using F-score in feature selection

F-score can be used to measure the discrimination of two sets of real-numbers and can be used for feature selection. However, I once read that A disadvantage of F-score is that it does not reveal ...
0
votes
0answers
12 views

How to deal with missing data when calculating Information Gain

While working on a neural network for classification problem I'm dealing with huge number of possible features and information gain seems like a good way to narrow them down (there are hundreds of ...
0
votes
1answer
24 views

Feature importance in gradient boosted trees

I am tuning the parameters of a gradient boosting regression tree algorithm and find it hard to understand the importance of some variables. Here is the case.. when the number of estimators is ...
31
votes
6answers
1k views

Variable selection for predictive modeling really needed in 2016?

This question has been asked on CV some yrs ago, it seems worth a repost in light of 1) order of magnitude better computing technology (e.g. parallel computing, HPC etc) and 2) newer techniques, e.g. [...
2
votes
1answer
56 views

Challenges in interpretation of variable selection from LASSO and OLS [duplicate]

I work as a consultant and I am often faced with variable selection and prediction problems. For my clients, I run OLS and I am recently pushing for penalized methods which can handle variable ...
2
votes
2answers
407 views

How to cross validate stepwise logistic regression?

I have a conceptual problem understanding how to cross validate stepwise logistic regression. Every time the training set is divided it is very likely that different features are chosen based on the ...
0
votes
0answers
14 views

Can LARS or Coordinate Descent select features that are marginally uncorrelated with the response?

I could construct a response Y the following way: Given $\left\lbrace X_k \right\rbrace_{k=1}^p$, and the regression model $$ Y = \sum_{i = 1}^p X_i - \rho p \beta_{p+1}X_{p+1} + \varepsilon,$$ if ...
0
votes
1answer
191 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
0
votes
1answer
20 views

Testing for feature importance with missing values

I'm looking for an appropriate model to do the following analysis: I'd like to test which courses are the most important in determining if a student stays in or leaves a university program. Imagine I ...
1
vote
0answers
16 views

Covariance-residual technique for linear regression feature selection

When doing forward feature selection for linear regression, it is a well known trick that to select the next feature to add, we can compute the covariance of each candidate feature against the current ...
1
vote
1answer
26 views

FInding relevant features for a time series segmentation

I have a time series data, where each of the data point belongs to one of the known clusters. What I am interested is to perform a HMM so that we can obtain hidden states that further abstracts out ...
0
votes
1answer
26 views

Selecting Subsets with Linear Correlation

I'm looking for a method of grouping 200+ samples with 30+ features into groups which share linear correlations among a subset of the features. I've found Ransac to sometimes return a good ...
0
votes
0answers
13 views

How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...
2
votes
2answers
634 views

Event Prediction through Machine Learning

I have a large data set consisting of ca. 40 categorical data items and a few interval data items (real numbers, less than 5 such items). Most categories should have a lot of values that repeat ...
2
votes
1answer
43 views

Can I use output of classifier A as feature for classifier B?

This is likely to be a confused question, but I'm curious if this is a valid way to combine classifiers. I have a classification data set, i.e. column of labels and N columns of features, and I use a ...
1
vote
1answer
218 views

Python Text Classification Features Engineering

I am trying to train a model on text classification. I have a large labeled dataset. Documents are set of comments, notes on a incident. Labels are high level categories for the incidents. As ...
0
votes
0answers
41 views

Recursive feature elimination and class imbalance

I am trying to apply the recursive feature elimination in the R package caret following the example in the caret website: ...
5
votes
1answer
39 views

What are some of the best approaches for variable selection in Poisson regression?

My target variable follows a Poisson distribution. I have to make a selection of best variables out of about 2000 variables. Is there any method exist for variable selection for poisson type ...
1
vote
2answers
68 views

Feature selection with the help of Genetic algorithm in datasets with small number of features and samples

As a project, i should perform feature selection on small unbalanced datasets with at most 30 features and also at most 300 samples with the help of Genetic algorithm. So, the chromosomes in GA are ...
2
votes
1answer
204 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
2
votes
1answer
20 views

Extension to SAFE screening rule for Lasso

In El Ghaoui et al. (2010), "Safe feature elimination in sparse learning" and following works, screening rules are derived for Lasso (as well as other L1-penalized problems): $ \min_w \|y-X w\|^2 + \...
0
votes
1answer
72 views

Using non-significant variables in model

I am trying to build a credit scoring model and have discovered and interesting approach for feature selection. I am looping through all features and removing them one by one (using variable ...
6
votes
3answers
400 views

What are the advantages of stepwise regression?

I am experimenting with stepwise regression for the sake of diversity in my approach to the problem. So, I have 2 questions: What are the advantages of stepwise regression? What are its specific ...
0
votes
1answer
27 views

In feature selection, are there any rules on choosing metrics to mesure relevance? (MI / Fisher score / correlation coefficient, etc)

This is a rather general question. If the question is vague and hard to answer in a few lines, I'd be happy if someone just point me to some readings. Thanks in Advance. I am working on a multi-class ...
0
votes
1answer
25 views

number of features in feature selection for text mining problems

Let's say for a text mining problem (e.g creating a predictive model using text analysis), using a feature selection method (e.g TF-IDF) we come up with 1000 features/words/tokens. Is there some ...
0
votes
1answer
222 views

MATLAB function TreeBagger() (Random Forest classification) and different number of variables

I am using MATLAB function TreeBagger() for Random Forest classification, for an assignment. It gives error when the number of variables of the Test data is different from the number of variables of ...
3
votes
1answer
277 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
1
vote
0answers
25 views

Comparing and ranking differentiating attributes across groups

I'm looking for some help on how to approach this problem. Say I have two or more groups of people. Each group has characteristics and attributes. For example, say we have the following two groups: ...
0
votes
1answer
184 views

combining multiple classifiers common features

Can multiple binary-classifiers be combined to produce a final output if their feature sets have some common elements? How will this influence the accuracy?
1
vote
1answer
70 views

Does PCA do something else apart from selecting features with the most variance?

While experimenting with Spark library MlLib, I questioned myself if I understood well the mechanism of PCA algorithm, because output of MlLib algorithm was not what I expected to get. so for given ...
0
votes
1answer
127 views

Feature selection with a binary dependent variable

Given we have a binary dependent variable and 100s of features and ~50k observations, is there a generally accepted way to trim the features via some type of machine learning concept? I was trying a ...
0
votes
2answers
29 views

choosing a model after feature selection process

so ive been selecting features for a regression problem and have obtained a list of the best performing feature sets. (note my list is actually several thousand lines long) 188.493 186.989 [379.45, 0....