Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

8
votes
0answers
1k views

How do you select variables in a regression model?

The traditional approach to variable selection is to find variables that contribute the most to predicting a new response. Recently I learned of an alternative to this. In modeling variables that ...
5
votes
0answers
784 views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
4
votes
0answers
69 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
153 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
4
votes
0answers
602 views

How exactly does Chi-square feature selection work?

I know that for each feature-class pair, the value of the chi-square statistic is computed and compared against a threshold. I am a little confused though. If there are $m$ features and $k$ classes, ...
3
votes
0answers
154 views

When would I choose Lasso over Elastic Net

What are the scenarios where Lasso is likely to perform better than Elastic Net (out of sample prediction)?
3
votes
0answers
88 views

Variable reduction by means of ANOVA?

I have a typical problem with several variables and a large amount of data which are not important right now. The goal of the study is to relate variable $Y$ with variables $X_1,X_2,...,X_n$. I have ...
3
votes
0answers
188 views

Microarray data: suggestions on Feature selection + Model training scheme?

I have a microarray expression dataset (46 samples, thousands of attributes) and I want to perform feature selection first, and, based on this subset of features (shouldn't be more than 4 or 5, based ...
3
votes
0answers
318 views

Variable selection / Dataset reduction for large datasets (in R)

I'm working on a behavoural scorecard modelling exercise, and many of the decisions taken to date have been based on the experience of a consulting credit analyst (whose experience software-wise is ...
3
votes
0answers
149 views

How does LASSO select among collinear predictors?

I'm looking for an intuitive answer why a GLM LASSO model selects a specific predictor out of a group of highly correlated ones, and why it does so differently then the best subset feature selection. ...
3
votes
0answers
144 views

Fast algorithm for variable selection

The (training) data contains 1280 observations with 1415 features. The test set has additional 380 observations. The data is sparse, that is, many of the variables has many zeros and few positive ...
3
votes
0answers
167 views

Sensible to include ratio as a variable in logistic regression?

I'm creating a generalised linear regression using a binomial link function for two variables A and B. From looking at the data it appears that A/B may have discriminatory effect. Is it sensible to ...
2
votes
0answers
32 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
2
votes
0answers
39 views

Feature selection: permutation test Vs deleting a variable

In feature selection for predictive models, it is usually applied a permutation test. In this test, all the values of one variable are randomly permuted and the prediction accuracy is extracted for ...
2
votes
0answers
36 views

Quantify the information lost given by the Kullback-Leibler divergence measure

Consider there are $N$ individuals and these measure a quantity $X\in \mathbb{R}^{N\times M}$ where $M$ is the number of measurements and let $P(X)$ denote a probability distribution over $X$. The ...
2
votes
0answers
32 views

Lasso-ing the order of a lag?

Suppose I have longitudinal data of the form $\mathbf Y = (Y_1, \ldots, Y_J) \sim \mathcal N(\mu, \Sigma)$ (I have multiple observations, this is just the form of a single one). I'm interested in ...
2
votes
0answers
69 views

Confusion related to feature selection

Well my objective is to predict solar energy radiation at a particular location given some features like wind, temperature, humidity ... I have a total data for 10 years where I have the measurement ...
2
votes
0answers
62 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
2
votes
0answers
29 views

Correct order of performing imputation and variable selection

This is a general question about performing data analysis. I have a data set with ~1000 sample size and 200 features. Some of features have more than 50% missing or even higher. The missing pattern is ...
2
votes
0answers
242 views

Feature importance scores of SVM multiclass one-vs-one design

Info about dataset: 5 classes, 200 trials, 100 features. (I know about the trial to feature ratio being very low, but can not avoid this here and still got well enough classification results.) ...
2
votes
0answers
39 views

Why do correlations matter in simulating data to compare classifiers when p >> N?

In genomics and computational biology, expression data sets contain a much larger number of features (p) than the number of observations (N). I wanted to simulate data where p>>N to compare the ...
2
votes
0answers
218 views

LASSO vs AIC for feature selection with the Cox model

I have some questions about the Lasso. After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, ...
2
votes
0answers
226 views

Kernel in PenalizedSVM R package

There is not option to select kernel in penalizedSVM R package. What kernel do they use? Is there some other R package with penalized SVM methods where I can choose various kernels?
2
votes
0answers
88 views

Random forest like techniques (bagging, random feature subset) for SGD methods

Are there any well-known results/tools/literature on using bagging and random feature subset selection for regression or SGD-based methods?
2
votes
0answers
176 views

Procedure for variable selection + logistic regression when n is small, p is large, and data are unbalanced?

I have data that have been collected using case-control procedures, in which the population of positive cases is collected with a random sample of negative cases. This yields 62 positive cases and 179 ...
2
votes
0answers
112 views

Feature construction for text mining

In the text mining, besides N-gram model, what are the state-of-art models for building feature space while capturing the dependence among the different words, or capturing the semantic meaning in the ...
2
votes
0answers
126 views

Non-linear (e.g. RBF kernel) SVM with SCAD penalties implementation

Is there one? I think there's a penalizedSVM package in R but it looks to use a linear kernel. Can't quite tell from the documentation. If it's linear, is there a R package that lets me calculate the ...
2
votes
0answers
460 views

Error using rfe in caret package in R

I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
2
votes
0answers
111 views

New development in variable selection in clustering using MCMC?

The latest general framework I know in MCMC-based wrapper method(doing variable selection and clustering simultaneously) are the paper "Bayesian variable selection in clustering high-dimensional data" ...
2
votes
0answers
158 views

What should I do to compare different sets of data?

I am a beginner in statistics, and I want to learn machine learning :). Therefore, I have gathered some sample data to practice. But, the problem is I want to create a feature (or attribute), which is ...
1
vote
0answers
31 views

methods to perform feature selection in unsupervissed learning

What are the available methods/implementation in R/Python to discard/select unimportant/important features in data? My data does not have labels (unsupervised). The data has ~100 features with mixed ...
1
vote
0answers
29 views

Using Leave-One-Out Cross Validation with LARS

I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such: $$\mathbb{min}\ ...
1
vote
0answers
20 views

Setting up feature vectors

I am working on a classification project and I want to use SVM's and/or Clustering Algs. What I am having trouble with is deciding how to set up my feature vectors. I have already decided what my ...
1
vote
0answers
23 views

Unstable models, repeated crossvalidation, feature selection

I'm still trying to classify few (about 200) samples in a high dimensional feature space (dim=19) into 3 (very unbalanced) classes. I use an implementation of Least Squares SVM with one vs one coding ...
1
vote
0answers
64 views

Is this normal to have big value of Chi Squared?

I am using chi squared for feature selection in text classification. However when I compute it I sometime have very big values. Like 100, 1000 or even 20000. Is this normal ? I wonder because I ...
1
vote
0answers
24 views

why feature scaling or weighting is important in surpervised learning?

I can understand feature scaling or weighting is important in unsurpervised learning case, because we want an good representation of "similarity". But why it is also important in surpervised learning ...
1
vote
0answers
14 views

Is linear kernel SVM performance between features indicative of RBF kernel SVM performance?

I have feature set 1 and feature set 2. If a linear kernel SVM performs better ("better" meaning greater classification accuracy) when using feature set 1, does this guarantee that a properly tuned ...
1
vote
0answers
58 views

Feature extraction for customer churn data

I have a customer churn data, and would be implementing algorithms (decision tree, logistic regression, segment analysis). I have doubt on feature extraction procedure though. The training sample has ...
1
vote
0answers
19 views

Is mRMR feature selection sensitive to imbalanced dataset?

I wish to perform feature selection on a dataset using mRMR but my classes are imbalanced roughly 3:1. Should I down sample the majority class or use the whole dataset? If I use the whole dataset will ...
1
vote
0answers
38 views

How to determine appropriate number of features and also which features to select?

So I have a dataset which I am using K fold cross validation on to select the number of features and which features should be selected. As I understand it, I would set the number of features to be ...
1
vote
0answers
37 views

What are some classic examples of feature selection in classification?

Is there a classic example showing the importance of good feature selection in classification? The ideal example would be simple, and very easy to understand. I've been volunteered/instructed to put ...
1
vote
0answers
44 views

Why do Laplacian Eigenmaps ignore the second KKT condition in the solution calculation?

From the original Laplacian Eigenmaps paper, we find the new manifold embedding $Y$ as follows, given the Laplacian and degree matrices $L$ and $D$: $\arg\min_Ytr(Y^TLY)$ subject to ${Y^TDY=I}$. ...
1
vote
0answers
23 views

Clustering and Feature Selection For Audio Data

I'm very new to stats and I'm at a loss as to where to start with this problem. I'm not sure what tools or methods to use to extract meaningful answers from my data. I'll try and describe the problem ...
1
vote
0answers
36 views

Linear regression for feature selection

Imagine we regress y on x1...x4. Now, we want to find out if ...
1
vote
0answers
46 views

Music classification into genres

I need to classify music (songs) into genres (rock, french house, trash metal, etc). My idea was to extract features from the songs (bmp, zero crossing, etc) and then apply known classification ...
1
vote
0answers
55 views

Classifier predicts only one class

I was trying myself in kaggle CIFAR competition, I trained lots of classifiers but get the same result/fail (don't know how to treat them), maybe someone could help me figure what i'm doing wrong. ...
1
vote
0answers
27 views

Estimating confidence of a prediction

Given a set of features vectors $X=\{\vec{x}_1,..,\vec{x}_n\}$, binary ground truth data $Y=\{y_1,..,y_n\}$ and continuous prediction $\bar{Y} = \{\bar{y}_1,..,\bar{y}_n\}\in [0,1]$, I want to perform ...
1
vote
0answers
58 views

Feature selection methods comparison

I recently run a project that involves a feature selection step before further pattern recognition. The number of features for our data set is very large and instead of running greedy ...
1
vote
0answers
48 views

Inferring dimension weight in a mapping from a triangle to a distribution over its vertices

I have a dataset $(y_i, \mathbf{X}_i)$, where $\mathbf{X}_i$ is a $3 \times n$ matrix of reals and $y_i$ takes a value in $\{1, 2, 3\}$. Essentially, $y_i$ represents a "selection" of the row vector ...
1
vote
0answers
69 views

How to explain difference of importance between feature selection and model quality?

I have a data collection with a mixed feature set consisting of both numerical features and text features. The number of numerical features is quite small, i.e., 6, comparing to the number of text ...