Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

4
votes
0answers
302 views

When would I choose Lasso over Elastic Net

What are the scenarios where Lasso is likely to perform better than Elastic Net (out of sample prediction)?
4
votes
0answers
106 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
173 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
4
votes
0answers
707 views

How exactly does Chi-square feature selection work?

I know that for each feature-class pair, the value of the chi-square statistic is computed and compared against a threshold. I am a little confused though. If there are $m$ features and $k$ classes, ...
3
votes
0answers
35 views

Development data set for feature engineering and data exploration

I dont hear this being talked about much: If you want to engineer features and visually explore the data, should you do this on a development set separate from the training and test set? If ...
3
votes
0answers
76 views

LASSO prediction model question

I am trying to create a prediction model with 33 predictors (brain metabolite levels in various regions) and 8 observations (cognitive test scores) with p>>n problem using LASSO in MATLAB (...
3
votes
0answers
141 views

Variable reduction by means of ANOVA?

I have a typical problem with several variables and a large amount of data which are not important right now. The goal of the study is to relate variable $Y$ with variables $X_1,X_2,...,X_n$. I have ...
3
votes
0answers
104 views

Mutual information/pointwise mutual information for measuring prediction

I want to measure how well I predict a vector $Y$ (vector not a label) for observation $X$. Both $X$ and $Y$ have the same set of features ($1\times n$). For that, I thought of "scoring" the ...
3
votes
0answers
311 views

Microarray data: suggestions on Feature selection + Model training scheme?

I have a microarray expression dataset (46 samples, thousands of attributes) and I want to perform feature selection first, and, based on this subset of features (shouldn't be more than 4 or 5, based ...
3
votes
0answers
560 views

Variable selection / Dataset reduction for large datasets (in R)

I'm working on a behavoural scorecard modelling exercise, and many of the decisions taken to date have been based on the experience of a consulting credit analyst (whose experience software-wise is ...
3
votes
0answers
259 views

How does LASSO select among collinear predictors?

I'm looking for an intuitive answer why a GLM LASSO model selects a specific predictor out of a group of highly correlated ones, and why it does so differently then the best subset feature selection. ...
3
votes
0answers
203 views

Fast algorithm for variable selection

The (training) data contains 1280 observations with 1415 features. The test set has additional 380 observations. The data is sparse, that is, many of the variables has many zeros and few positive ...
3
votes
0answers
206 views

Sensible to include ratio as a variable in logistic regression?

I'm creating a generalised linear regression using a binomial link function for two variables A and B. From looking at the data it appears that A/B may have discriminatory effect. Is it sensible to ...
2
votes
0answers
25 views

Random Forest: Class specific feature importance

I'm using the bigrf R-package to analyse a dataset with ca. 50.000 observations x 120 variables, classified into two groups. After growing a forest of 1000 trees, ...
2
votes
0answers
43 views

How do I use weight vector of SVM and logistic regression for feature importance?

I have trained a SVM and logistic regression classifier on my dataset for binary classification. Both classifier provide a weight vector which is of the size of the number of features. I can use this ...
2
votes
0answers
21 views

Feature selection of SVM

My question is three-fold In the context of "Kernelized" support vector machines Is variable/feature selection desirable - especially since we regularize the parameter C to prevent overfitting and ...
2
votes
0answers
75 views

Generating features: What level of interaction?

I have multi (3) level data indexed by i,j,t. As such, I can generate fixed effects (dummies) for either ij, it, or jt, (and still achieve identification). I can also do i,j,t separately as well. ...
2
votes
0answers
49 views

How to make use of less data of a particular class for better modeling?

I have a dataset, say 9000 rows, with some features. Around 8000 belong to class 1 and 1000 to class 0. So, if I am creating a model with any method say SVM, LR, Random forest the model has a tendency ...
2
votes
0answers
102 views

Deep learning: representation learning or classification?

For classification, I have often heard about deep learning / deep neural networks as a form of representation learning. I am confused as to what "representation learning" means in this context. Which ...
2
votes
0answers
28 views

Can I reuse the dataset set aside for performing t-test based on the following condition?

I have a small number of samples and large number features. For doing the feature selection I'm going to divide my total set into a feature selection set and a test set.I run the t-test on the former ...
2
votes
0answers
66 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
2
votes
0answers
85 views

Quantify the information lost given by the Kullback-Leibler divergence measure

Consider there are $N$ individuals and these measure a quantity $X\in \mathbb{R}^{N\times M}$ where $M$ is the number of measurements and let $P(X)$ denote a probability distribution over $X$. The ...
2
votes
0answers
247 views

Linear regression for feature selection

Imagine we regress y on x1...x4. Now, we want to find out if ...
2
votes
0answers
94 views

Confusion related to feature selection

Well my objective is to predict solar energy radiation at a particular location given some features like wind, temperature, humidity ... I have a total data for 10 years where I have the measurement ...
2
votes
0answers
222 views

Features selection by filter methods for multivariate time series

I have a data set in which the samples are multivariate (about 30 variable/features) time series. These samples refer to two classes. I would like to select the variables more relevant to discriminate ...
2
votes
0answers
86 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
2
votes
0answers
41 views

Correct order of performing imputation and variable selection

This is a general question about performing data analysis. I have a data set with ~1000 sample size and 200 features. Some of features have more than 50% missing or even higher. The missing pattern is ...
2
votes
0answers
360 views

Feature importance scores of SVM multiclass one-vs-one design

Info about dataset: 5 classes, 200 trials, 100 features. (I know about the trial to feature ratio being very low, but can not avoid this here and still got well enough classification results.) ...
2
votes
0answers
44 views

Why do correlations matter in simulating data to compare classifiers when p >> N?

In genomics and computational biology, expression data sets contain a much larger number of features (p) than the number of observations (N). I wanted to simulate data where p>>N to compare the ...
2
votes
0answers
341 views

LASSO vs AIC for feature selection with the Cox model

I have some questions about the Lasso. After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, ...
2
votes
0answers
334 views

Kernel in PenalizedSVM R package

There is not option to select kernel in penalizedSVM R package. What kernel do they use? Is there some other R package with penalized SVM methods where I can choose various kernels?
2
votes
0answers
133 views

Random forest like techniques (bagging, random feature subset) for SGD methods

Are there any well-known results/tools/literature on using bagging and random feature subset selection for regression or SGD-based methods?
2
votes
0answers
276 views

Procedure for variable selection + logistic regression when n is small, p is large, and data are unbalanced?

I have data that have been collected using case-control procedures, in which the population of positive cases is collected with a random sample of negative cases. This yields 62 positive cases and 179 ...
2
votes
0answers
171 views

Non-linear (e.g. RBF kernel) SVM with SCAD penalties implementation

Is there one? I think there's a penalizedSVM package in R but it looks to use a linear kernel. Can't quite tell from the documentation. If it's linear, is there a R package that lets me calculate the ...
2
votes
0answers
739 views

Error using rfe in caret package in R

I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
2
votes
0answers
121 views

New development in variable selection in clustering using MCMC?

The latest general framework I know in MCMC-based wrapper method(doing variable selection and clustering simultaneously) are the paper "Bayesian variable selection in clustering high-dimensional data" ...
2
votes
0answers
162 views

What should I do to compare different sets of data?

I am a beginner in statistics, and I want to learn machine learning :). Therefore, I have gathered some sample data to practice. But, the problem is I want to create a feature (or attribute), which is ...
1
vote
0answers
14 views

How to use RFECV for feature selection and cross validation

I am still very new to machine learning and trying to figure things out myself. I am using SciKit learn and have a data set of tweets with around 20,000 features (n_features=20,000). So far I achieved ...
1
vote
0answers
13 views

What is the general procedure or general rules for grouping factor levels?

I am attempting to build a predictive (machine-learning) logistic regression model that contains mostly categorical (non-ordinal) variables. As part of a variable selection process I run a Pearson ...
1
vote
0answers
14 views

SVM and concatenation of features

For example I train my SVM on 3 features set of different size: Training data size [rows cols]: features1 [nsamples 100] features2 [nsamples 50] features3 [nsamples 128] And for each feature set ...
1
vote
0answers
21 views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the important of each feature for each pair of classes ...
1
vote
0answers
29 views

How can I extract features from fMRI network connectivity analysis (FSL nets)?

I have a set of 37 fMRI images from mice which are divided into 4 classes (different drug doses applied). My task is to train classifiers (SVM etc.) on this dataset. Of course feature extraction is a ...
1
vote
0answers
29 views

random forest analysis with random(clustering) varible

My data consists of presence/absence (PA) of a trait in 354 plants collected from 127 collection sites as response, and a set of 25 climatic continuous variables in each site as predictors. The ...
1
vote
0answers
48 views

Calculating Mutual Information for feature selection

In order to determine the importance of some individual features coming from labelled time series, I am trying to calculate the Mutual Information (as showed in "Who do you sync you are?: smartphone ...
1
vote
0answers
51 views

chi square test for large data sets

I use the Chi-square test for feature selection. I use it only when all entries in the contingency table are greater then 5. Is that the correct approach statistically? What happens for example, if ...
1
vote
0answers
25 views

Initial screening of variables

I recently came across an approach where the long list of potential predictors (around 100+) is screened for its explanatory power and basis this initial screening, a smaller set of predictors is ...
1
vote
0answers
31 views

How to determine if a categorical variable affects an independent variable?

I am looking at the wine dataset from the R FactoMineRpackage to find out whether the categorical variable soil type affects the ...
1
vote
0answers
13 views

Does adding a grouping variable to a model help or hamper?

Imagine I have a dataset of people where I can find the city and country they live in. The data is such that, given the city, there is only one possible country. For example, given Madrid as a city, ...
1
vote
0answers
94 views

What kind of feature selection can Chi square test be used for?

Here I am asking about what others commonly do to use chi squared test for feature selection wrt outcome in supervised learning. If I understand correctly, do they test the independence between ...
1
vote
0answers
22 views

Number of candidate inputs that can be handled by different modelling techniques

Am I correct if I say that some modelling techniques can handle better a larger number of candidate inputs? (If we hold the number of observation constant). Let's say I put around 60 different ...