Methods and principles of selecting a subset of attributes for use in further modelling
3
votes
0answers
506 views
How exactly does Chi-square feature selection work?
I know that for each feature-class pair, the value of the chi-square statistic is computed and compared against a threshold.
I am a little confused though. If there are $m$ features and $k$ classes, ...
2
votes
1answer
225 views
Open source implementation elastic net in C or C++
Can anyone provide or point me to a freely available implemention of Elastic Net in C or C++?
3
votes
4answers
525 views
How to know when to stop reducing dimensions with PCA?
I'm using PCA to reduce dimensionality before I feed the data into a classifier. My bootstrap/cross-validation has shown a significant reduction in test error as a result of applying PCA and keeping ...
2
votes
1answer
281 views
How to select the final model with elastic net feature selection, cross validation and SVM?
I have a dataset of some 100 samples, each with >10,000 features, some of which highly correlated. Here's what I am doing currently.
Split the data set into three folds.
For each fold,
2.1 Run ...
2
votes
0answers
82 views
Non-linear (e.g. RBF kernel) SVM with SCAD penalties implementation
Is there one? I think there's a penalizedSVM package in R but it looks to use a linear kernel. Can't quite tell from the documentation. If it's linear, is there a R package that lets me calculate the ...
1
vote
3answers
646 views
Use of PCA analysis to selection of variables for a regression analysis
I have too many environmental variables to use in a multiple regression analysis. If I use all the variables to models are just to complex. The use of the PCA axis in the regression analysis was ...
2
votes
1answer
778 views
R knn variable selection
I have a data set that's 200k rows X 50 columns. I'm trying to use a knn model on it but there is huge variance in performance depending on which variables are ...
6
votes
1answer
198 views
Dealing with very large time-series datasets
I have access to a very large dataset. The data is from MEG recordings of people listening to musical excerpts, from one of four genres. The data is as follows:
6 Subjects
3 Experimental repetitions ...
6
votes
1answer
188 views
Automatic feature selection for anomaly detection
What is the best way to automatically select features for anomaly detection?
I normally treat Anomaly Detection as an algorithm where the features are selected by human experts: what matters is the ...
0
votes
0answers
66 views
How to organize a series of data analysis results?
For several days, I collected a series of data analysis results for given data with different statistical methods, different sample size, different other analysis parameters, and so on.
So right now, ...
0
votes
2answers
545 views
How to integrate principal components with GLM?
How would I integrate the output of a principal components analysis with a GLM (assuming the PCA is used for variable selection for the GLM)?
9
votes
2answers
165 views
Domain-agnostic feature engineering that retains semantic meaning?
Feature engineering is often an important component to machine learning (it was used heavily to win the KDD Cup in 2010). However, I find that most feature engineering techniques either
destroy ...
0
votes
1answer
26 views
Inter-feature ratio explicitly or implicitly?
Let's say I have two numerical features were is suspect that the ratio between them is the most meaningful way of looking at them. I have a NN learner.
Should I add the ratio as a third feature or is ...
7
votes
2answers
2k views
Is there a way to use cross validation to do variable/feature selection in R?
I have a data set with about 70 variables that I'd like to cut down. What I'm looking to do is use CV to find most useful variables in the following fashion.
1) Randomly select say 20 variables.
...
3
votes
1answer
263 views
Reference for random forests
I would like to understand how do the Boruta package work. Could you suggest some references for the theoretical aspect of so-called random forests? Thanks.
Below are two illustrative examples of why ...
7
votes
2answers
483 views
Best approach for model selection Bayesian or cross-validation?
When trying to select among various models or the number of features to include for, say prediction I can think of two approaches.
Split the data into training and test sets. Better still, use ...
1
vote
1answer
336 views
The disadvantage of using F-score in feature selection
F-score can be used to measure the discrimination of two sets of real-numbers and can be used for feature selection. However, I once read that
A disadvantage of F-score is that it does not reveal ...
12
votes
3answers
1k views
Variables are often adjusted (e.g. standardised) before making a model - when is this a good idea, and when is it a bad one?
In what circumstances would you want to, or not want to scale or standardize a variable prior to model fitting? And what are the advantages / disadvantages of scaling a variable?
3
votes
1answer
359 views
Which are the most effective methods for selecting independent variables?
Some clustering algorithms require independence of variables but (especially working with real data) variables are often highly correlated.
I have been suggested to apply a Principal Component ...
12
votes
3answers
816 views
Why is variable selection necessary?
Common data-based variable selection procedures (for example, forward, backward, stepwise, all subsets) tend to yield models with undesirable properties, including:
Coefficients biased away from ...
3
votes
1answer
426 views
Text feature vector extraction
I have a class assignment to implement a couple existing ways to extract feature vectors from a given set of texts, so they can be used to classify those texts using k-nearest neighbour algorithm. The ...
4
votes
1answer
211 views
Feature selection with k-fold cross-validated least angle regression
I am using the least angle regression (LARS) to extract the most important predictors ($x_1, x_2,...,x_p$) for my response variable ($y$).
I have seven predictors ($x_1,x_2,...,x_7$) for each ...
12
votes
2answers
887 views
Significance testing or cross validation?
Two common approaches for selecting correlated variables are significance tests and cross validation. What problem does each try to solve and when would I prefer one over the other?
2
votes
1answer
161 views
How to know which variables are more important in a process? [closed]
I have a process with 15 effective variables. I could record 9 variables to study its effect on process.
I am looking for an appropriate factor to estimate the value of effectiveness of each factor. I ...
2
votes
2answers
86 views
Analyzing the added effect of an individual variable having fitted one variable already, using a generalized linear model
I am using a generalized linear model to analyse my data, there are 6 explanatory variables. I have added all different variable combinations in different models, and ranked them according to their ...
0
votes
3answers
524 views
How to identify suitable variables to assess confounding, mediation and effect modification?
Imagine that you are planning a study about risk behaviours among HIV positive injecting drug users.
All the individuals included in the sample are injecting drugs and all are HIV positive. The main ...
2
votes
2answers
55 views
3
votes
1answer
83 views
High dimensional volume entropy estimator
I am writing a program using high-dimensional volume (HDV) estimator to estimate entropy and mutual information for variable selection. Let $ D = (x^i_1, x^i_2, ..., x^i_M)$, N is the number of data ...
3
votes
1answer
166 views
Choosing variables for Discriminant Analysis
I've 110 variables & 200 data points. Of this 110 variables, one is group variable (say "brown eye","blue eye"). I want to use discriminant analysis to classify the groups based on remaining 119 ...
7
votes
3answers
2k views
Improving the SVM classification of diabetes
I am using SVM to predict diabetes. I am using the BRFSS data set for this purpose. The data set has the dimensions of $432607 \times 136$ and is skewed. The percentage of ...
2
votes
1answer
558 views
Best way to select useful features using R software
I have a huge matrix (individuals X features with row.names as individuals numbers) and the corresponding segment in another vector of 1D (row.names are the same as in my huge matrix and the vector ...
2
votes
2answers
372 views
Variables importance: who can do the most pushups?
I don't know enough math to formulate an intelligent question on this so I'll give an example. I'd like an answer to my example but also I'd like to know the jargon I need to be able to research it ...
4
votes
2answers
262 views
Feature selection for low probability event prediction
I'm currently trying to predict the probability for low probability events (~1%).
I have large DB with ~200,000 vectors (~2000 plus examples) with ~200 features.
I'm trying to find the the best ...
3
votes
0answers
449 views
Clustering probability distributions - methods & metrics?
I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
2
votes
2answers
205 views
Feature selection and latent variables
I would like to know if it is useful (or maybe dangerous) to reduce the number of attributes (by selecting the most informative ones among thousands) before seeking for latent variables or not (in an ...
2
votes
0answers
296 views
Error using rfe in caret package in R
I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
3
votes
4answers
615 views
Proper variable selection method for glm
I have a mixed model with a continuous outcome variable and a certain number of predictors. Some need to be included in the model no matter what (sex, age, and a "main factor"), and others must be ...
1
vote
0answers
119 views
Measures of predictive power of attributes in data mining
What are the most widely used measures of predictive power of attributes in scoring models?
Motivation: I have a lot of attributes, more than I can study by myself and I want to select somehow the ...
1
vote
1answer
190 views
Variable selection for increasing accuracy
I know that there are various posts regarding variable selection but I am asking something particular. With respect to the question that I posted today in the following link:
Low accuracy in out of ...
2
votes
0answers
94 views
New development in variable selection in clustering using MCMC?
The latest general framework I know in MCMC-based wrapper method(doing variable selection and clustering simultaneously) are the paper "Bayesian variable selection in clustering high-dimensional data" ...
9
votes
1answer
426 views
Random permutation test for feature selection
I am confused about permutation analysis for feature selection in a logistic regression context.
Could you provide a clear explanation of the random permutation test and how does it applies to feature ...
2
votes
4answers
236 views
Suggestions for identifying key features
I have a large set of customer data. For these customers, I have devised a customer loyalty score which is a measure of the loyalty of the customer. I want to find the features that are strongly ...
3
votes
1answer
286 views
How can I assess how descriptive feature vectors are?
I am assessing how good different features are for unsupervised classification of a set of objects. For each different feature I test, I have computed a feature vector that describes the object. I ...
0
votes
1answer
159 views
2
votes
1answer
176 views
Interpretation of “one” feature change in a supervised classifier
i'm making experiments using app. 5000 labeled dataset.i'm trying different supervised ML algorithm to evaluate the results.The vector size is 13 with the labels (totally 12 features+1 label) and i ...
8
votes
3answers
843 views
The use of median polish for feature selection
In a paper I was reading recently I came across the following bit in their data analysis section:
The data table was then split into tissues and cell lines, and the two subtables were separately ...
14
votes
3answers
2k views
What are disadvantages of using the lasso for variable selection for regression?
From what I know, using lasso for variable selection handles the problem of correlated inputs. Also, since it is equivalent to Least Angle Regression, it is not slow computationally. However, many ...
9
votes
2answers
405 views
Best methods of feature selection for nonparametric regression
A newbie question here. I am currently performing a nonparametric regression using the np package in R. I have 7 features and using a brute force approach I identified the best 3. But, soon I will ...
4
votes
4answers
131 views
Feature selection for classification, controlling for sub-population
I have a bunch of points that belong to one of population P1, P2, ... Pn AND to class A or B.
Within each population I'll be doing classification between A and B, and I want to select features that ...
7
votes
1answer
142 views
How to quantify redundancy of features?
I have three features that I use to solve a classification problem. Originally, these features produced boolean values, so I could evaluate their redundancy by looking at how much the sets of positive ...
