Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
1answer
32 views

Inter-feature ratio explicitly or implicitly?

Let's say I have two numerical features were is suspect that the ratio between them is the most meaningful way of looking at them. I have a NN learner. Should I add the ratio as a third feature or is ...
8
votes
4answers
3k views

Is there a way to use cross validation to do variable/feature selection in R?

I have a data set with about 70 variables that I'd like to cut down. What I'm looking to do is use CV to find most useful variables in the following fashion. 1) Randomly select say 20 variables. ...
3
votes
1answer
444 views

Reference for random forests

I would like to understand how do the Boruta package work. Could you suggest some references for the theoretical aspect of so-called random forests? Thanks. Below are two illustrative examples of why ...
12
votes
2answers
1k views

Best approach for model selection Bayesian or cross-validation?

When trying to select among various models or the number of features to include for, say prediction I can think of two approaches. Split the data into training and test sets. Better still, use ...
2
votes
2answers
1k views

The disadvantage of using F-score in feature selection

F-score can be used to measure the discrimination of two sets of real-numbers and can be used for feature selection. However, I once read that A disadvantage of F-score is that it does not reveal ...
27
votes
3answers
6k views

Variables are often adjusted (e.g. standardised) before making a model - when is this a good idea, and when is it a bad one?

In what circumstances would you want to, or not want to scale or standardize a variable prior to model fitting? And what are the advantages / disadvantages of scaling a variable?
3
votes
1answer
474 views

Which are the most effective methods for selecting independent variables?

Some clustering algorithms require independence of variables but (especially working with real data) variables are often highly correlated. I have been suggested to apply a Principal Component ...
20
votes
3answers
2k views

Why is variable selection necessary?

Common data-based variable selection procedures (for example, forward, backward, stepwise, all subsets) tend to yield models with undesirable properties, including: Coefficients biased away from ...
3
votes
1answer
838 views

Text feature vector extraction

I have a class assignment to implement a couple existing ways to extract feature vectors from a given set of texts, so they can be used to classify those texts using k-nearest neighbour algorithm. The ...
4
votes
1answer
362 views

Feature selection with k-fold cross-validated least angle regression

I am using the least angle regression (LARS) to extract the most important predictors ($x_1, x_2,...,x_p$) for my response variable ($y$). I have seven predictors ($x_1,x_2,...,x_7$) for each ...
16
votes
2answers
2k views

Significance testing or cross validation?

Two common approaches for selecting correlated variables are significance tests and cross validation. What problem does each try to solve and when would I prefer one over the other?
2
votes
1answer
187 views

How to know which variables are more important in a process? [closed]

I have a process with 15 effective variables. I could record 9 variables to study its effect on process. I am looking for an appropriate factor to estimate the value of effectiveness of each factor. I ...
2
votes
2answers
96 views

Analyzing the added effect of an individual variable having fitted one variable already, using a generalized linear model

I am using a generalized linear model to analyse my data, there are 6 explanatory variables. I have added all different variable combinations in different models, and ranked them according to their ...
1
vote
3answers
769 views

How to identify suitable variables to assess confounding, mediation and effect modification?

Imagine that you are planning a study about risk behaviours among HIV positive injecting drug users. All the individuals included in the sample are injecting drugs and all are HIV positive. The main ...
8
votes
5answers
3k views

Can I use PCA to do variable selection for cluster analysis?

I have to reduce the number of variables to conduct a cluster analysis. My variables are strongly correlated, so I thought to do a Factor Analysis PCA (principal component analysis). However, if I use ...
2
votes
2answers
82 views
3
votes
1answer
113 views

High dimensional volume entropy estimator

I am writing a program using high-dimensional volume (HDV) estimator to estimate entropy and mutual information for variable selection. Let $ D = (x^i_1, x^i_2, ..., x^i_M)$, N is the number of data ...
4
votes
1answer
314 views

Choosing variables for Discriminant Analysis

I've 110 variables & 200 data points. Of this 110 variables, one is group variable (say "brown eye","blue eye"). I want to use discriminant analysis to classify the groups based on remaining 119 ...
9
votes
3answers
5k views

Improving the SVM classification of diabetes

I am using SVM to predict diabetes. I am using the BRFSS data set for this purpose. The data set has the dimensions of $432607 \times 136$ and is skewed. The percentage of ...
2
votes
1answer
1k views

Best way to select useful features using R software

I have a huge matrix (individuals X features with row.names as individuals numbers) and the corresponding segment in another vector of 1D (row.names are the same as in my huge matrix and the vector ...
2
votes
2answers
490 views

Variables importance: who can do the most pushups?

I don't know enough math to formulate an intelligent question on this so I'll give an example. I'd like an answer to my example but also I'd like to know the jargon I need to be able to research it ...
5
votes
2answers
494 views

Feature selection for low probability event prediction

I'm currently trying to predict the probability for low probability events (~1%). I have large DB with ~200,000 vectors (~2000 plus examples) with ~200 features. I'm trying to find the the best ...
6
votes
1answer
1k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
3
votes
2answers
393 views

Feature selection and latent variables

I would like to know if it is useful (or maybe dangerous) to reduce the number of attributes (by selecting the most informative ones among thousands) before seeking for latent variables or not (in an ...
2
votes
0answers
606 views

Error using rfe in caret package in R

I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
3
votes
4answers
1k views

Proper variable selection method for glm

I have a mixed model with a continuous outcome variable and a certain number of predictors. Some need to be included in the model no matter what (sex, age, and a "main factor"), and others must be ...
1
vote
0answers
157 views

Measures of predictive power of attributes in data mining

What are the most widely used measures of predictive power of attributes in scoring models? Motivation: I have a lot of attributes, more than I can study by myself and I want to select somehow the ...
1
vote
1answer
223 views

Variable selection for increasing accuracy

I know that there are various posts regarding variable selection but I am asking something particular. With respect to the question that I posted today in the following link: Low accuracy in out of ...
2
votes
0answers
118 views

New development in variable selection in clustering using MCMC?

The latest general framework I know in MCMC-based wrapper method(doing variable selection and clustering simultaneously) are the paper "Bayesian variable selection in clustering high-dimensional data" ...
9
votes
1answer
695 views

Random permutation test for feature selection

I am confused about permutation analysis for feature selection in a logistic regression context. Could you provide a clear explanation of the random permutation test and how does it applies to feature ...
3
votes
4answers
292 views

Suggestions for identifying key features

I have a large set of customer data. For these customers, I have devised a customer loyalty score which is a measure of the loyalty of the customer. I want to find the features that are strongly ...
3
votes
1answer
556 views

How can I assess how descriptive feature vectors are?

I am assessing how good different features are for unsupervised classification of a set of objects. For each different feature I test, I have computed a feature vector that describes the object. I ...
0
votes
1answer
196 views

How to get scored combination of features

My data looks like this (F=Features) ...
2
votes
1answer
200 views

Interpretation of “one” feature change in a supervised classifier

i'm making experiments using app. 5000 labeled dataset.i'm trying different supervised ML algorithm to evaluate the results.The vector size is 13 with the labels (totally 12 features+1 label) and i ...
9
votes
3answers
2k views

The use of median polish for feature selection

In a paper I was reading recently I came across the following bit in their data analysis section: The data table was then split into tissues and cell lines, and the two subtables were separately ...
5
votes
2answers
1k views

Is it possible to use kernel PCA for feature selection?

Is it possible to use kernel principal component analysis (kPCA) for Latent Semantic Indexing (LSI) in the same way as PCA is used? I perform LSI in R using the prcomp PCA function and extract the ...
22
votes
3answers
5k views

What are disadvantages of using the lasso for variable selection for regression?

From what I know, using lasso for variable selection handles the problem of correlated inputs. Also, since it is equivalent to Least Angle Regression, it is not slow computationally. However, many ...
9
votes
2answers
602 views

Best methods of feature selection for nonparametric regression

A newbie question here. I am currently performing a nonparametric regression using the np package in R. I have 7 features and using a brute force approach I identified the best 3. But, soon I will ...
4
votes
4answers
150 views

Feature selection for classification, controlling for sub-population

I have a bunch of points that belong to one of population P1, P2, ... Pn AND to class A or B. Within each population I'll be doing classification between A and B, and I want to select features that ...
8
votes
1answer
196 views

How to quantify redundancy of features?

I have three features that I use to solve a classification problem. Originally, these features produced boolean values, so I could evaluate their redundancy by looking at how much the sets of positive ...
2
votes
0answers
160 views

What should I do to compare different sets of data?

I am a beginner in statistics, and I want to learn machine learning :). Therefore, I have gathered some sample data to practice. But, the problem is I want to create a feature (or attribute), which is ...
0
votes
0answers
218 views

What are the most relevant metrics for social games, and how are they calculated?

For example some common ones are engagement, churn, ARPU. The problem with the metrics I mentioned (besides needen to know more) is that I do not understand clearly how are they measure and what they ...
9
votes
3answers
3k views

How to apply LASSO to IRLS (logistic regression)?

I have programmed a logistic regression using the IRLS algorithm. I would like to apply a LASSO penalization in order to automatically select the right features. At each iteration, the following is ...
2
votes
1answer
66 views

Analysis of variables of varying numbers

I work with amino acid sequences and I want to use a self-made model to tell me something about it, lets call it $f(\text{seq})$. Now i want to know the contribution of every position in the sequence ...
8
votes
2answers
2k views

Soft-thresholding vs. Lasso penalization

I am trying to summarize what I understood so far in penalized multivariate analysis with high-dimensional data sets, and I still struggle through getting a proper definition of soft-thresholding vs. ...
1
vote
2answers
214 views

How best to reduce dimensionality of a dataset composed of events and trials?

I'm trying to reduce dataset dimensionality. PCA is a good metric but that gives me new dataset. My goal is to determine from number of events (e.g. 60) and number of trials (e.g. 6) which events are ...
31
votes
8answers
5k views

Feature selection for “final” model when performing cross-validation in machine learning

I am getting a bit confused about feature selection and machine learning and I was wondering if you could help me out. I have a microarray dataset that is classified into two groups and has 1000s of ...
18
votes
2answers
3k views

Variable importance from SVM

How to obtain a variable (attribute) importance using SVM?
8
votes
3answers
1k views

Computing best subset of predictors for linear regression

For the selection of predictors in multivariate linear regression with $p$ suitable predictors, what methods are available to find an 'optimal' subset of the predictors without explicitly testing all ...
10
votes
4answers
2k views

Application of machine learning techniques in small sample clinical studies

What do you think about applying machine learning techniques, like Random Forests or penalized regression (with L1 or L2 penalty, or a combination thereof) in small sample clinical studies when the ...