Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

4
votes
0answers
279 views

When would I choose Lasso over Elastic Net

What are the scenarios where Lasso is likely to perform better than Elastic Net (out of sample prediction)?
4
votes
0answers
101 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
168 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
4
votes
0answers
698 views

How exactly does Chi-square feature selection work?

I know that for each feature-class pair, the value of the chi-square statistic is computed and compared against a threshold. I am a little confused though. If there are $m$ features and $k$ classes, ...
3
votes
0answers
29 views

Development data set for feature engineering and data exploration

I dont hear this being talked about much: If you want to engineer features and visually explore the data, should you do this on a development set separate from the training and test set? If ...
3
votes
0answers
72 views

LASSO prediction model question

I am trying to create a prediction model with 33 predictors (brain metabolite levels in various regions) and 8 observations (cognitive test scores) with p>>n problem using LASSO in MATLAB (...
3
votes
0answers
132 views

Variable reduction by means of ANOVA?

I have a typical problem with several variables and a large amount of data which are not important right now. The goal of the study is to relate variable $Y$ with variables $X_1,X_2,...,X_n$. I have ...
3
votes
0answers
99 views

Mutual information/pointwise mutual information for measuring prediction

I want to measure how well I predict a vector $Y$ (vector not a label) for observation $X$. Both $X$ and $Y$ have the same set of features ($1\times n$). For that, I thought of "scoring" the ...
3
votes
0answers
289 views

Microarray data: suggestions on Feature selection + Model training scheme?

I have a microarray expression dataset (46 samples, thousands of attributes) and I want to perform feature selection first, and, based on this subset of features (shouldn't be more than 4 or 5, based ...
3
votes
0answers
526 views

Variable selection / Dataset reduction for large datasets (in R)

I'm working on a behavoural scorecard modelling exercise, and many of the decisions taken to date have been based on the experience of a consulting credit analyst (whose experience software-wise is ...
3
votes
0answers
241 views

How does LASSO select among collinear predictors?

I'm looking for an intuitive answer why a GLM LASSO model selects a specific predictor out of a group of highly correlated ones, and why it does so differently then the best subset feature selection. ...
3
votes
0answers
199 views

Fast algorithm for variable selection

The (training) data contains 1280 observations with 1415 features. The test set has additional 380 observations. The data is sparse, that is, many of the variables has many zeros and few positive ...
3
votes
0answers
204 views

Sensible to include ratio as a variable in logistic regression?

I'm creating a generalised linear regression using a binomial link function for two variables A and B. From looking at the data it appears that A/B may have discriminatory effect. Is it sensible to ...
2
votes
0answers
70 views

Generating features: What level of interaction?

I have multi (3) level data indexed by i,j,t. As such, I can generate fixed effects (dummies) for either ij, it, or jt, (and still achieve identification). I can also do i,j,t separately as well. ...
2
votes
0answers
38 views

How to make use of less data of a particular class for better modeling?

I have a dataset, say 9000 rows, with some features. Around 8000 belong to class 1 and 1000 to class 0. So, if I am creating a model with any method say SVM, LR, Random forest the model has a tendency ...
2
votes
0answers
81 views

Deep learning: representation learning or classification?

For classification, I have often heard about deep learning / deep neural networks as a form of representation learning. I am confused as to what "representation learning" means in this context. Which ...
2
votes
0answers
25 views

Can I reuse the dataset set aside for performing t-test based on the following condition?

I have a small number of samples and large number features. For doing the feature selection I'm going to divide my total set into a feature selection set and a test set.I run the t-test on the former ...
2
votes
0answers
62 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
2
votes
0answers
80 views

Quantify the information lost given by the Kullback-Leibler divergence measure

Consider there are $N$ individuals and these measure a quantity $X\in \mathbb{R}^{N\times M}$ where $M$ is the number of measurements and let $P(X)$ denote a probability distribution over $X$. The ...
2
votes
0answers
92 views

Confusion related to feature selection

Well my objective is to predict solar energy radiation at a particular location given some features like wind, temperature, humidity ... I have a total data for 10 years where I have the measurement ...
2
votes
0answers
204 views

Features selection by filter methods for multivariate time series

I have a data set in which the samples are multivariate (about 30 variable/features) time series. These samples refer to two classes. I would like to select the variables more relevant to discriminate ...
2
votes
0answers
83 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
2
votes
0answers
36 views

Correct order of performing imputation and variable selection

This is a general question about performing data analysis. I have a data set with ~1000 sample size and 200 features. Some of features have more than 50% missing or even higher. The missing pattern is ...
2
votes
0answers
342 views

Feature importance scores of SVM multiclass one-vs-one design

Info about dataset: 5 classes, 200 trials, 100 features. (I know about the trial to feature ratio being very low, but can not avoid this here and still got well enough classification results.) ...
2
votes
0answers
44 views

Why do correlations matter in simulating data to compare classifiers when p >> N?

In genomics and computational biology, expression data sets contain a much larger number of features (p) than the number of observations (N). I wanted to simulate data where p>>N to compare the ...
2
votes
0answers
326 views

LASSO vs AIC for feature selection with the Cox model

I have some questions about the Lasso. After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, ...
2
votes
0answers
314 views

Kernel in PenalizedSVM R package

There is not option to select kernel in penalizedSVM R package. What kernel do they use? Is there some other R package with penalized SVM methods where I can choose various kernels?
2
votes
0answers
123 views

Random forest like techniques (bagging, random feature subset) for SGD methods

Are there any well-known results/tools/literature on using bagging and random feature subset selection for regression or SGD-based methods?
2
votes
0answers
275 views

Procedure for variable selection + logistic regression when n is small, p is large, and data are unbalanced?

I have data that have been collected using case-control procedures, in which the population of positive cases is collected with a random sample of negative cases. This yields 62 positive cases and 179 ...
2
votes
0answers
167 views

Non-linear (e.g. RBF kernel) SVM with SCAD penalties implementation

Is there one? I think there's a penalizedSVM package in R but it looks to use a linear kernel. Can't quite tell from the documentation. If it's linear, is there a R package that lets me calculate the ...
2
votes
0answers
700 views

Error using rfe in caret package in R

I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
2
votes
0answers
120 views

New development in variable selection in clustering using MCMC?

The latest general framework I know in MCMC-based wrapper method(doing variable selection and clustering simultaneously) are the paper "Bayesian variable selection in clustering high-dimensional data" ...
2
votes
0answers
162 views

What should I do to compare different sets of data?

I am a beginner in statistics, and I want to learn machine learning :). Therefore, I have gathered some sample data to practice. But, the problem is I want to create a feature (or attribute), which is ...
1
vote
0answers
15 views

random forest analysis with random(clustering) varible

My data consists of presence/absence (PA) of a trait in 354 plants collected from 127 collection sites as response, and a set of 25 climatic continuous variables in each site as predictors. The ...
1
vote
0answers
37 views

Calculating Mutual Information for feature selection

In order to determine the importance of some individual features coming from labelled time series, I am trying to calculate the Mutual Information (as showed in "Who do you sync you are?: smartphone ...
1
vote
0answers
32 views

chi square test for large data sets

I use the Chi-square test for feature selection. I use it only when all entries in the contingency table are greater then 5. Is that the correct approach statistically? What happens for example, if ...
1
vote
0answers
31 views

How to determine if a categorical variable affects an independent variable?

I am looking at the wine dataset from the R FactoMineRpackage to find out whether the categorical variable soil type affects the ...
1
vote
0answers
11 views

Does adding a grouping variable to a model help or hamper?

Imagine I have a dataset of people where I can find the city and country they live in. The data is such that, given the city, there is only one possible country. For example, given Madrid as a city, ...
1
vote
0answers
18 views

Number of candidate inputs that can be handled by different modelling techniques

Am I correct if I say that some modelling techniques can handle better a larger number of candidate inputs? (If we hold the number of observation constant). Let's say I put around 60 different ...
1
vote
0answers
33 views

Feature selection in clustering

I am looking for a method for feature selection in Gaussian Mixture Models. I have a dataset with 2000 records and 40 variables. I tried to use the "clustvarsel" package in R, which use the BIC as ...
1
vote
0answers
19 views

Looking for a principled/systematic procedure for discarding features

I have a collection of $M_i \times N$ matrices $X_i$ whose rows are (raw) feature vectors (from a common $N$-dimensional feature space). MATLAB reports that most of the covariance matrices $C_i := ...
1
vote
0answers
45 views

Including Feature Selection in Cross Validation - Application to Bag of Words

I am working on a prediction problem where I was given a 6,000 record dataset with the value of the dependent variable included ("train"), and a 2,000 record dataset with the same independent ...
1
vote
0answers
12 views

Feature Extraction for landscape images

I work on a landscape-image database (forests, beaches, landmarks, cities etc.) and I'm trying to find the similarity between them, I don't have labels or any specific classes, I just want to find ...
1
vote
0answers
15 views

Relation between chi-squared statistic scores and classification accuracy

I am evaluating the utility of two distinct sets of features for solving a given supervised classification problem with two classes. I am using the chi-squared statistic as a feature selection ...
1
vote
0answers
31 views

High dimensional explanatory variable

I have a data set of 22 observations and 6931 variables. Data belongs to two classes, 0 and 1. I would like to know which features are important for each class (species) and which one contribute the ...
1
vote
0answers
27 views

Learning if instances from a dataset are part of the same subset

I was wondering if there are some well-known machine learning methodologies for subset learning. In other words, to learn if two instances are part of the same subset or not (boolean label?). One ...
1
vote
0answers
11 views

Need a statistic for comparing “strength” of Markov blankets in a Bayesian network

Working with Bayesian networks. I take a given network structure and fit its parameters on data. I am looking for a statistic based on those parameter estimates that allows me to compare Markov ...
1
vote
0answers
24 views

feature selection in a small sample size

I need an advice. I have a dataset consisting of 108 observations (27 subjects * 4 time points) and ~10000 features. The data represents intensity values (comes from continuous domain). When I run ...
1
vote
0answers
12 views

Will the classification accuracy vary if we first classify based on a single variable and then use the rest?

Let's suppose I am doing classification and that I have 99 features and another feature that says if the person is male or female. I have two options viz to build one classifier using all the ...
1
vote
0answers
43 views

How to reduce the number of features for Gaussian Process regression?

Ridge regression reduces complexity of the model by scaling down the coefficient. Lasso reduces the complexity of the model by selecting the features used. For Gaussian Process, is there similar way ...