Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

-1
votes
1answer
111 views

High-dimensional dependent binary explanatory variable

I am dealing with a data set containing roughly $n=4000$ binary observations $Y_1, \ldots, Y_n$ with $p=1000$ binary explanatory variables. I suspect that a lot of these explanatory variables are not ...
1
vote
1answer
341 views

SVM importance of predictor variables

I am building a model in R using support vector machine (SVM) with KBF kernel. The model seems to work quite well. I would like to assess the relative importance of predictor variables. Can anyone ...
3
votes
1answer
1k views

Feature selection methods for document classtification

I have a simple document classification problem where i need to classify some documents to a definite set of classes. I need to perform a feature selection (where I will select the most important ...
1
vote
0answers
56 views

What are some common tools, initial approaches to data in a prediction problem when facing too many predictors?

If one is given several hundred features (of both categorical and continuous type) what are some approaches to determining which features to keep or even drop? Data as such is difficult to visualize ...
2
votes
1answer
129 views

Feature selection without target variable

Let's assume I have a NxD matrix X with the N rows being observations and the D columns being features. I would now like to know which are the most "interesting" features of this dataset. I.e. which ...
1
vote
1answer
250 views

Feature selection weighting 2 filters in Naive Bayes

I am trying to do text classification using Naive Bayes. Before training, I would like to make feature selection in order to reduce the feature space dimension. In order to do so, I have thought of ...
12
votes
2answers
9k views

How to deal with multicollinearity when performing variable selection?

I have a dataset with 9 continuous independent variables. I'm trying to select amongst these variables to fit a model to a single percentage (dependent) variable, ...
1
vote
1answer
347 views

Is there a rule of thumb on the relationship between the number of instances and the number of features?

If we build a classifier based on a very small number of instances (say, fewer than 300) and the number of features we are using is very large (say, larger than 100k features). If we decide to ...
4
votes
2answers
1k views

Feature selection for the text mining?

Before performing the task of text mining, we need to select the features for characterizing each given document. Are there any systematic guidance on choosing the document features? How does the ...
0
votes
1answer
147 views

Statistics regarding different data sets having a common response variable

Lets assume one has 2 datasets: with different number of rows (samples) and columns (features). Each of these 2 datasets have a column as a binary response variable. Lets say healthy or not. What sort ...
4
votes
1answer
251 views

Is this a correct procedure for feature selection using cross-validation?

I was looking for information about feature selection and crossvalidation, when I found this post: Feature selection for "final" model when performing cross-validation in machine learning. ...
3
votes
2answers
290 views

Feature selection for disease classification based on tests

I have a dataset of around 100 different subjects Some of them have a disease, some do not (roughly 60:40 disease:no disease) They are subjected to a battery of 15 tests, to see if they are outside ...
1
vote
0answers
204 views

References about univariable vs multivariable variable selection

Suppose I have variables $X_j$, $j=1,\ldots,p$, some of which are correlated, and some continuous output $y$. I want to rank the variables by importance. One way is to do an association test of each ...
2
votes
4answers
3k views

How to convert nominal dataset into numerical dataset?

For my work, im using the multilabel dataset from this webpage. Few dataset which are listed in the page (for, e.g bibtex) have nominal attributes, i.e attribute values are 0 and 1. My queries are ...
4
votes
0answers
679 views

How exactly does Chi-square feature selection work?

I know that for each feature-class pair, the value of the chi-square statistic is computed and compared against a threshold. I am a little confused though. If there are $m$ features and $k$ classes, ...
2
votes
1answer
498 views

Open source implementation elastic net in C or C++

Can anyone provide or point me to a freely available implemention of Elastic Net in C or C++?
4
votes
4answers
937 views

How to know when to stop reducing dimensions with PCA?

I'm using PCA to reduce dimensionality before I feed the data into a classifier. My bootstrap/cross-validation has shown a significant reduction in test error as a result of applying PCA and keeping ...
4
votes
1answer
788 views

How to select the final model with elastic net feature selection, cross validation and SVM?

I have a dataset of some 100 samples, each with >10,000 features, some of which highly correlated. Here's what I am doing currently. Split the data set into three folds. For each fold, 2.1 Run ...
2
votes
0answers
162 views

Non-linear (e.g. RBF kernel) SVM with SCAD penalties implementation

Is there one? I think there's a penalizedSVM package in R but it looks to use a linear kernel. Can't quite tell from the documentation. If it's linear, is there a R package that lets me calculate the ...
1
vote
3answers
2k views

Use of PCA analysis to select variables for a regression analysis [duplicate]

I have too many environmental variables to use in a multiple regression analysis. If I use all the variables the models are just too complex. The use of the PCA axes in the regression analysis was ...
3
votes
1answer
2k views

R knn variable selection

I have a data set that's 200k rows X 50 columns. I'm trying to use a knn model on it but there is huge variance in performance depending on which variables are ...
8
votes
1answer
573 views

Dealing with very large time-series datasets

I have access to a very large dataset. The data is from MEG recordings of people listening to musical excerpts, from one of four genres. The data is as follows: 6 Subjects 3 Experimental repetitions ...
7
votes
1answer
507 views

Automatic feature selection for anomaly detection

What is the best way to automatically select features for anomaly detection? I normally treat Anomaly Detection as an algorithm where the features are selected by human experts: what matters is the ...
0
votes
0answers
83 views

How to organize a series of data analysis results?

For several days, I collected a series of data analysis results for given data with different statistical methods, different sample size, different other analysis parameters, and so on. So right now, ...
4
votes
3answers
1k views

How to use principal components as predictors in GLM?

How would I use the output of a principal components analysis (PCA) in a generalized linear model (GLM), assuming the PCA is used for variable selection for the GLM? Clarification: I want to use PCA ...
9
votes
2answers
329 views

Domain-agnostic feature engineering that retains semantic meaning?

Feature engineering is often an important component to machine learning (it was used heavily to win the KDD Cup in 2010). However, I find that most feature engineering techniques either destroy ...
0
votes
1answer
32 views

Inter-feature ratio explicitly or implicitly?

Let's say I have two numerical features were is suspect that the ratio between them is the most meaningful way of looking at them. I have a NN learner. Should I add the ratio as a third feature or is ...
8
votes
4answers
3k views

Is there a way to use cross validation to do variable/feature selection in R?

I have a data set with about 70 variables that I'd like to cut down. What I'm looking to do is use CV to find most useful variables in the following fashion. 1) Randomly select say 20 variables. ...
3
votes
1answer
478 views

Reference for random forests

I would like to understand how do the Boruta package work. Could you suggest some references for the theoretical aspect of so-called random forests? Thanks. Below are two illustrative examples of why ...
12
votes
2answers
1k views

Best approach for model selection Bayesian or cross-validation?

When trying to select among various models or the number of features to include for, say prediction I can think of two approaches. Split the data into training and test sets. Better still, use ...
2
votes
2answers
1k views

The disadvantage of using F-score in feature selection

F-score can be used to measure the discrimination of two sets of real-numbers and can be used for feature selection. However, I once read that A disadvantage of F-score is that it does not reveal ...
29
votes
3answers
6k views

Variables are often adjusted (e.g. standardised) before making a model - when is this a good idea, and when is it a bad one?

In what circumstances would you want to, or not want to scale or standardize a variable prior to model fitting? And what are the advantages / disadvantages of scaling a variable?
3
votes
1answer
482 views

Which are the most effective methods for selecting independent variables?

Some clustering algorithms require independence of variables but (especially working with real data) variables are often highly correlated. I have been suggested to apply a Principal Component ...
21
votes
3answers
2k views

Why is variable selection necessary?

Common data-based variable selection procedures (for example, forward, backward, stepwise, all subsets) tend to yield models with undesirable properties, including: Coefficients biased away from ...
3
votes
1answer
864 views

Text feature vector extraction

I have a class assignment to implement a couple existing ways to extract feature vectors from a given set of texts, so they can be used to classify those texts using k-nearest neighbour algorithm. The ...
4
votes
1answer
370 views

Feature selection with k-fold cross-validated least angle regression

I am using the least angle regression (LARS) to extract the most important predictors ($x_1, x_2,...,x_p$) for my response variable ($y$). I have seven predictors ($x_1,x_2,...,x_7$) for each ...
16
votes
2answers
2k views

Significance testing or cross validation?

Two common approaches for selecting correlated variables are significance tests and cross validation. What problem does each try to solve and when would I prefer one over the other?
2
votes
1answer
188 views

How to know which variables are more important in a process? [closed]

I have a process with 15 effective variables. I could record 9 variables to study its effect on process. I am looking for an appropriate factor to estimate the value of effectiveness of each factor. I ...
2
votes
2answers
96 views

Analyzing the added effect of an individual variable having fitted one variable already, using a generalized linear model

I am using a generalized linear model to analyse my data, there are 6 explanatory variables. I have added all different variable combinations in different models, and ranked them according to their ...
1
vote
3answers
785 views

How to identify suitable variables to assess confounding, mediation and effect modification?

Imagine that you are planning a study about risk behaviours among HIV positive injecting drug users. All the individuals included in the sample are injecting drugs and all are HIV positive. The main ...
8
votes
5answers
4k views

Can I use PCA to do variable selection for cluster analysis?

I have to reduce the number of variables to conduct a cluster analysis. My variables are strongly correlated, so I thought to do a Factor Analysis PCA (principal component analysis). However, if I use ...
2
votes
2answers
82 views
3
votes
1answer
113 views

High dimensional volume entropy estimator

I am writing a program using high-dimensional volume (HDV) estimator to estimate entropy and mutual information for variable selection. Let $ D = (x^i_1, x^i_2, ..., x^i_M)$, N is the number of data ...
4
votes
1answer
319 views

Choosing variables for Discriminant Analysis

I've 110 variables & 200 data points. Of this 110 variables, one is group variable (say "brown eye","blue eye"). I want to use discriminant analysis to classify the groups based on remaining 119 ...
9
votes
3answers
5k views

Improving the SVM classification of diabetes

I am using SVM to predict diabetes. I am using the BRFSS data set for this purpose. The data set has the dimensions of $432607 \times 136$ and is skewed. The percentage of ...
2
votes
1answer
1k views

Best way to select useful features using R software

I have a huge matrix (individuals X features with row.names as individuals numbers) and the corresponding segment in another vector of 1D (row.names are the same as in my huge matrix and the vector ...
2
votes
2answers
497 views

Variables importance: who can do the most pushups?

I don't know enough math to formulate an intelligent question on this so I'll give an example. I'd like an answer to my example but also I'd like to know the jargon I need to be able to research it ...
5
votes
2answers
505 views

Feature selection for low probability event prediction

I'm currently trying to predict the probability for low probability events (~1%). I have large DB with ~200,000 vectors (~2000 plus examples) with ~200 features. I'm trying to find the the best ...
7
votes
1answer
1k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
3
votes
2answers
393 views

Feature selection and latent variables

I would like to know if it is useful (or maybe dangerous) to reduce the number of attributes (by selecting the most informative ones among thousands) before seeking for latent variables or not (in an ...