Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

4
votes
0answers
75 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
154 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
4
votes
0answers
622 views

How exactly does Chi-square feature selection work?

I know that for each feature-class pair, the value of the chi-square statistic is computed and compared against a threshold. I am a little confused though. If there are $m$ features and $k$ classes, ...
3
votes
0answers
165 views

When would I choose Lasso over Elastic Net

What are the scenarios where Lasso is likely to perform better than Elastic Net (out of sample prediction)?
3
votes
0answers
92 views

Variable reduction by means of ANOVA?

I have a typical problem with several variables and a large amount of data which are not important right now. The goal of the study is to relate variable $Y$ with variables $X_1,X_2,...,X_n$. I have ...
3
votes
0answers
81 views

Mutual information/pointwise mutual information for measuring prediction

I want to measure how well I predict a vector $Y$ (vector not a label) for observation $X$. Both $X$ and $Y$ have the same set of features ($1\times n$). For that, I thought of "scoring" the ...
3
votes
0answers
198 views

Microarray data: suggestions on Feature selection + Model training scheme?

I have a microarray expression dataset (46 samples, thousands of attributes) and I want to perform feature selection first, and, based on this subset of features (shouldn't be more than 4 or 5, based ...
3
votes
0answers
339 views

Variable selection / Dataset reduction for large datasets (in R)

I'm working on a behavoural scorecard modelling exercise, and many of the decisions taken to date have been based on the experience of a consulting credit analyst (whose experience software-wise is ...
3
votes
0answers
159 views

How does LASSO select among collinear predictors?

I'm looking for an intuitive answer why a GLM LASSO model selects a specific predictor out of a group of highly correlated ones, and why it does so differently then the best subset feature selection. ...
3
votes
0answers
151 views

Fast algorithm for variable selection

The (training) data contains 1280 observations with 1415 features. The test set has additional 380 observations. The data is sparse, that is, many of the variables has many zeros and few positive ...
3
votes
0answers
172 views

Sensible to include ratio as a variable in logistic regression?

I'm creating a generalised linear regression using a binomial link function for two variables A and B. From looking at the data it appears that A/B may have discriminatory effect. Is it sensible to ...
2
votes
0answers
37 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
2
votes
0answers
48 views

Feature selection: permutation test Vs deleting a variable

In feature selection for predictive models, it is usually applied a permutation test. In this test, all the values of one variable are randomly permuted and the prediction accuracy is extracted for ...
2
votes
0answers
44 views

Quantify the information lost given by the Kullback-Leibler divergence measure

Consider there are $N$ individuals and these measure a quantity $X\in \mathbb{R}^{N\times M}$ where $M$ is the number of measurements and let $P(X)$ denote a probability distribution over $X$. The ...
2
votes
0answers
72 views

Confusion related to feature selection

Well my objective is to predict solar energy radiation at a particular location given some features like wind, temperature, humidity ... I have a total data for 10 years where I have the measurement ...
2
votes
0answers
66 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
2
votes
0answers
31 views

Correct order of performing imputation and variable selection

This is a general question about performing data analysis. I have a data set with ~1000 sample size and 200 features. Some of features have more than 50% missing or even higher. The missing pattern is ...
2
votes
0answers
258 views

Feature importance scores of SVM multiclass one-vs-one design

Info about dataset: 5 classes, 200 trials, 100 features. (I know about the trial to feature ratio being very low, but can not avoid this here and still got well enough classification results.) ...
2
votes
0answers
39 views

Why do correlations matter in simulating data to compare classifiers when p >> N?

In genomics and computational biology, expression data sets contain a much larger number of features (p) than the number of observations (N). I wanted to simulate data where p>>N to compare the ...
2
votes
0answers
237 views

LASSO vs AIC for feature selection with the Cox model

I have some questions about the Lasso. After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, ...
2
votes
0answers
243 views

Kernel in PenalizedSVM R package

There is not option to select kernel in penalizedSVM R package. What kernel do they use? Is there some other R package with penalized SVM methods where I can choose various kernels?
2
votes
0answers
94 views

Random forest like techniques (bagging, random feature subset) for SGD methods

Are there any well-known results/tools/literature on using bagging and random feature subset selection for regression or SGD-based methods?
2
votes
0answers
182 views

Procedure for variable selection + logistic regression when n is small, p is large, and data are unbalanced?

I have data that have been collected using case-control procedures, in which the population of positive cases is collected with a random sample of negative cases. This yields 62 positive cases and 179 ...
2
votes
0answers
115 views

Feature construction for text mining

In the text mining, besides N-gram model, what are the state-of-art models for building feature space while capturing the dependence among the different words, or capturing the semantic meaning in the ...
2
votes
0answers
131 views

Non-linear (e.g. RBF kernel) SVM with SCAD penalties implementation

Is there one? I think there's a penalizedSVM package in R but it looks to use a linear kernel. Can't quite tell from the documentation. If it's linear, is there a R package that lets me calculate the ...
2
votes
0answers
489 views

Error using rfe in caret package in R

I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
2
votes
0answers
112 views

New development in variable selection in clustering using MCMC?

The latest general framework I know in MCMC-based wrapper method(doing variable selection and clustering simultaneously) are the paper "Bayesian variable selection in clustering high-dimensional data" ...
2
votes
0answers
158 views

What should I do to compare different sets of data?

I am a beginner in statistics, and I want to learn machine learning :). Therefore, I have gathered some sample data to practice. But, the problem is I want to create a feature (or attribute), which is ...
1
vote
0answers
18 views

Feature boosting via rescaling in logistic regression and linear SVMs

If I were expressing a problem in terms of binary features, all encoded as {0,1}, could I boost some features by encoding them as {0,2}? Would the effect change based on whether I used either of the ...
1
vote
0answers
10 views

Does within-group heterogeneity negatively impact random forest classification?

I have two rather conceptual questions about random forest classifiers. Before we get there, I quickly want to lay out the problem I am working on: I have large a large data set consisting of 300 ...
1
vote
0answers
32 views

categorical feature ranking

I would like to rank categorical features by the order or importance in a classification/regression setting. Input There are two features, which are survey questions: "how is your mood?": four ...
1
vote
0answers
17 views

How to assess the importance of the features which come from intersection of features of the two models?

I have two models from two different data sets. Model 1 contain 50 features and model 2 contain 40 features. the intersection of features of model 1 and 2 is 10. so how can I assess the relative ...
1
vote
0answers
36 views

Using Leave-One-Out Cross Validation with LARS

I have a kind of obscure question about using the Least Angle Regression (LARS) algorithm for variable selection. If I'm understanding it right, my professor formulates LARS as such: $$\mathbb{min}\ ...
1
vote
0answers
21 views

Setting up feature vectors

I am working on a classification project and I want to use SVM's and/or Clustering Algs. What I am having trouble with is deciding how to set up my feature vectors. I have already decided what my ...
1
vote
0answers
33 views

Unstable models, repeated crossvalidation, feature selection

I'm still trying to classify few (about 200) samples in a high dimensional feature space (dim=19) into 3 (very unbalanced) classes. I use an implementation of Least Squares SVM with one vs one coding ...
1
vote
0answers
67 views

Is this normal to have big value of Chi Squared?

I am using chi squared for feature selection in text classification. However when I compute it I sometime have very big values. Like 100, 1000 or even 20000. Is this normal ? I wonder because I ...
1
vote
0answers
50 views

How to apply feature selection based on tf-idf threshold

Let's say we have the following matrix (typical VSM example): ...
1
vote
0answers
28 views

why feature scaling or weighting is important in surpervised learning?

I can understand feature scaling or weighting is important in unsurpervised learning case, because we want an good representation of "similarity". But why it is also important in surpervised learning ...
1
vote
0answers
19 views

Is linear kernel SVM performance between features indicative of RBF kernel SVM performance?

I have feature set 1 and feature set 2. If a linear kernel SVM performs better ("better" meaning greater classification accuracy) when using feature set 1, does this guarantee that a properly tuned ...
1
vote
0answers
67 views

Feature extraction for customer churn data

I have a customer churn data, and would be implementing algorithms (decision tree, logistic regression, segment analysis). I have doubt on feature extraction procedure though. The training sample has ...
1
vote
0answers
11 views

What kinds of features are worth taking from objects on images?

I managed to label and get set of pixels for every single object of interest in the image. Now I'm looking into clustering and am not sure what features are worth using. I currently have two in mind ...
1
vote
0answers
21 views

Is mRMR feature selection sensitive to imbalanced dataset?

I wish to perform feature selection on a dataset using mRMR but my classes are imbalanced roughly 3:1. Should I down sample the majority class or use the whole dataset? If I use the whole dataset will ...
1
vote
0answers
41 views

How to determine appropriate number of features and also which features to select?

So I have a dataset which I am using K fold cross validation on to select the number of features and which features should be selected. As I understand it, I would set the number of features to be ...
1
vote
0answers
40 views

What are some classic examples of feature selection in classification?

Is there a classic example showing the importance of good feature selection in classification? The ideal example would be simple, and very easy to understand. I've been volunteered/instructed to put ...
1
vote
0answers
49 views

Why do Laplacian Eigenmaps ignore the second KKT condition in the solution calculation?

From the original Laplacian Eigenmaps paper, we find the new manifold embedding $Y$ as follows, given the Laplacian and degree matrices $L$ and $D$: $\arg\min_Ytr(Y^TLY)$ subject to ${Y^TDY=I}$. ...
1
vote
0answers
27 views

Clustering and Feature Selection For Audio Data

I'm very new to stats and I'm at a loss as to where to start with this problem. I'm not sure what tools or methods to use to extract meaningful answers from my data. I'll try and describe the problem ...
1
vote
0answers
41 views

Linear regression for feature selection

Imagine we regress y on x1...x4. Now, we want to find out if ...
1
vote
0answers
54 views

Music classification into genres

I need to classify music (songs) into genres (rock, french house, trash metal, etc). My idea was to extract features from the songs (bmp, zero crossing, etc) and then apply known classification ...
1
vote
0answers
64 views

Classifier predicts only one class

I was trying myself in kaggle CIFAR competition, I trained lots of classifiers but get the same result/fail (don't know how to treat them), maybe someone could help me figure what i'm doing wrong. ...
1
vote
0answers
30 views

Estimating confidence of a prediction

Given a set of features vectors $X=\{\vec{x}_1,..,\vec{x}_n\}$, binary ground truth data $Y=\{y_1,..,y_n\}$ and continuous prediction $\bar{Y} = \{\bar{y}_1,..,\bar{y}_n\}\in [0,1]$, I want to perform ...