Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

4
votes
0answers
247 views

When would I choose Lasso over Elastic Net

What are the scenarios where Lasso is likely to perform better than Elastic Net (out of sample prediction)?
4
votes
0answers
92 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
162 views

Variable Selection One by One vs Simultaneously

The high dimensional variable selection problem is really popular now. But I have a question: If I do simple linear regression regressing one response variable on 1 covariate at a time first and then ...
4
votes
0answers
677 views

How exactly does Chi-square feature selection work?

I know that for each feature-class pair, the value of the chi-square statistic is computed and compared against a threshold. I am a little confused though. If there are $m$ features and $k$ classes, ...
3
votes
0answers
22 views

Development data set for feature engineering and data exploration

I dont hear this being talked about much: If you want to engineer features and visually explore the data, should you do this on a development set separate from the training and test set? If ...
3
votes
0answers
57 views

Placement of earlier features in more complex features in CNN

I'm trying to understand convolutional neural networks better. I've been doing different tutorials, but there are some basics concerning how the hidden units represents features that I really would ...
3
votes
0answers
63 views

LASSO prediction model question

I am trying to create a prediction model with 33 predictors (brain metabolite levels in various regions) and 8 observations (cognitive test scores) with p>>n problem using LASSO in MATLAB (...
3
votes
0answers
119 views

Variable reduction by means of ANOVA?

I have a typical problem with several variables and a large amount of data which are not important right now. The goal of the study is to relate variable $Y$ with variables $X_1,X_2,...,X_n$. I have ...
3
votes
0answers
94 views

Mutual information/pointwise mutual information for measuring prediction

I want to measure how well I predict a vector $Y$ (vector not a label) for observation $X$. Both $X$ and $Y$ have the same set of features ($1\times n$). For that, I thought of "scoring" the ...
3
votes
0answers
262 views

Microarray data: suggestions on Feature selection + Model training scheme?

I have a microarray expression dataset (46 samples, thousands of attributes) and I want to perform feature selection first, and, based on this subset of features (shouldn't be more than 4 or 5, based ...
3
votes
0answers
462 views

Variable selection / Dataset reduction for large datasets (in R)

I'm working on a behavoural scorecard modelling exercise, and many of the decisions taken to date have been based on the experience of a consulting credit analyst (whose experience software-wise is ...
3
votes
0answers
209 views

How does LASSO select among collinear predictors?

I'm looking for an intuitive answer why a GLM LASSO model selects a specific predictor out of a group of highly correlated ones, and why it does so differently then the best subset feature selection. ...
3
votes
0answers
188 views

Fast algorithm for variable selection

The (training) data contains 1280 observations with 1415 features. The test set has additional 380 observations. The data is sparse, that is, many of the variables has many zeros and few positive ...
3
votes
0answers
196 views

Sensible to include ratio as a variable in logistic regression?

I'm creating a generalised linear regression using a binomial link function for two variables A and B. From looking at the data it appears that A/B may have discriminatory effect. Is it sensible to ...
2
votes
0answers
34 views

How to make use of less data of a particular class for better modeling?

I have a dataset, say 9000 rows, with some features. Around 8000 belong to class 1 and 1000 to class 0. So, if I am creating a model with any method say SVM, LR, Random forest the model has a tendency ...
2
votes
0answers
49 views

Deep learning: representation learning or classification?

For classification, I have often heard about deep learning / deep neural networks as a form of representation learning. I am confused as to what "representation learning" means in this context. Which ...
2
votes
0answers
43 views

How to increase the performance of random forest classifier?

I have a text classification task. These are the metrics for different languages at present: class1: 0.6823 class2: 0.7450 class3: 0.66 class4: 0.6719 How can I ...
2
votes
0answers
23 views

Can I reuse the dataset set aside for performing t-test based on the following condition?

I have a small number of samples and large number features. For doing the feature selection I'm going to divide my total set into a feature selection set and a test set.I run the t-test on the former ...
2
votes
0answers
57 views

category selection with LASSO

Suppose one has two features: color = {R, G, B} and t-shirt size = {S, M, L} and wants to regress these features on the probability of a sale, call it p. So the model is p ~ color + size. Now, the ...
2
votes
0answers
65 views

Feature selection: permutation test Vs deleting a variable

In feature selection for predictive models, it is usually applied a permutation test. In this test, all the values of one variable are randomly permuted and the prediction accuracy is extracted for ...
2
votes
0answers
70 views

Quantify the information lost given by the Kullback-Leibler divergence measure

Consider there are $N$ individuals and these measure a quantity $X\in \mathbb{R}^{N\times M}$ where $M$ is the number of measurements and let $P(X)$ denote a probability distribution over $X$. The ...
2
votes
0answers
85 views

Confusion related to feature selection

Well my objective is to predict solar energy radiation at a particular location given some features like wind, temperature, humidity ... I have a total data for 10 years where I have the measurement ...
2
votes
0answers
188 views

Features selection by filter methods for multivariate time series

I have a data set in which the samples are multivariate (about 30 variable/features) time series. These samples refer to two classes. I would like to select the variables more relevant to discriminate ...
2
votes
0answers
78 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
2
votes
0answers
33 views

Correct order of performing imputation and variable selection

This is a general question about performing data analysis. I have a data set with ~1000 sample size and 200 features. Some of features have more than 50% missing or even higher. The missing pattern is ...
2
votes
0answers
324 views

Feature importance scores of SVM multiclass one-vs-one design

Info about dataset: 5 classes, 200 trials, 100 features. (I know about the trial to feature ratio being very low, but can not avoid this here and still got well enough classification results.) ...
2
votes
0answers
43 views

Why do correlations matter in simulating data to compare classifiers when p >> N?

In genomics and computational biology, expression data sets contain a much larger number of features (p) than the number of observations (N). I wanted to simulate data where p>>N to compare the ...
2
votes
0answers
303 views

LASSO vs AIC for feature selection with the Cox model

I have some questions about the Lasso. After using the AIC or BIC to select a model, the model is fit with the variables selected in order to get the standard errors of the estimates with CIs, ...
2
votes
0answers
277 views

Kernel in PenalizedSVM R package

There is not option to select kernel in penalizedSVM R package. What kernel do they use? Is there some other R package with penalized SVM methods where I can choose various kernels?
2
votes
0answers
116 views

Random forest like techniques (bagging, random feature subset) for SGD methods

Are there any well-known results/tools/literature on using bagging and random feature subset selection for regression or SGD-based methods?
2
votes
0answers
271 views

Procedure for variable selection + logistic regression when n is small, p is large, and data are unbalanced?

I have data that have been collected using case-control procedures, in which the population of positive cases is collected with a random sample of negative cases. This yields 62 positive cases and 179 ...
2
votes
0answers
162 views

Non-linear (e.g. RBF kernel) SVM with SCAD penalties implementation

Is there one? I think there's a penalizedSVM package in R but it looks to use a linear kernel. Can't quite tell from the documentation. If it's linear, is there a R package that lets me calculate the ...
2
votes
0answers
635 views

Error using rfe in caret package in R

I am doing some exploratory data analysis in the Heritage Health Prize , and have come across a weird error using R's caret package. In the dataset, I've created a dataframe counting how many times a ...
2
votes
0answers
119 views

New development in variable selection in clustering using MCMC?

The latest general framework I know in MCMC-based wrapper method(doing variable selection and clustering simultaneously) are the paper "Bayesian variable selection in clustering high-dimensional data" ...
2
votes
0answers
161 views

What should I do to compare different sets of data?

I am a beginner in statistics, and I want to learn machine learning :). Therefore, I have gathered some sample data to practice. But, the problem is I want to create a feature (or attribute), which is ...
1
vote
0answers
30 views

giving some predictors higher priority in statistical models

i am working on statistical software debugging. i use various feature selection algorithms in order to discover the bug predictors in programs. for example regression coefficients indicate the ...
1
vote
0answers
13 views

Relation between chi-squared statistic scores and classification accuracy

I am evaluating the utility of two distinct sets of features for solving a given supervised classification problem with two classes. I am using the chi-squared statistic as a feature selection ...
1
vote
0answers
25 views

High dimensional explanatory variable

I have a data set of 22 observations and 6931 variables. Data belongs to two classes, 0 and 1. I would like to know which features are important for each class (species) and which one contribute the ...
1
vote
0answers
21 views

Learning if instances from a dataset are part of the same subset

I was wondering if there are some well-known machine learning methodologies for subset learning. In other words, to learn if two instances are part of the same subset or not (boolean label?). One ...
1
vote
0answers
17 views

feature selection in a small sample size

I need an advice. I have a dataset consisting of 108 observations (27 subjects * 4 time points) and ~10000 features. The data represents intensity values (comes from continuous domain). When I run ...
1
vote
0answers
12 views

Will the classification accuracy vary if we first classify based on a single variable and then use the rest?

Let's suppose I am doing classification and that I have 99 features and another feature that says if the person is male or female. I have two options viz to build one classifier using all the ...
1
vote
0answers
30 views

How to reduce the number of features for Gaussian Process regression?

Ridge regression reduces complexity of the model by scaling down the coefficient. Lasso reduces the complexity of the model by selecting the features used. For Gaussian Process, is there similar way ...
1
vote
0answers
21 views

Akaike information criterion for categorical and numerical data

How should I compute AIC for categorical and for numeric variables in classification problems? I see in Chapter 6 of Zumel and Mount that they use AIC before they train classification algorithms ...
1
vote
0answers
26 views

Poisson model with fractions

I have a simple website with a home page that has 5 different images on it. All images have a fixed set of 'features' associated with them (size, color, position etc.,). When a visitor comes to the ...
1
vote
0answers
16 views

Classification with two different dataset

I am working on a cancer classification model.Task is ,I am initially given a data set of 500 people and 1000 features.These people are given some kind of treatment(say Treatment 1). Some people are ...
1
vote
0answers
58 views

Different variable importance results with stabsel and mboost

I'm using glmboost in the mboost package to fit a boosted regression using linear models as the base learner. There are 13200 observations and about 75 variables, ...
1
vote
0answers
23 views

Regression to chose questions which better correlate with a 10 points likert like score

We have a survey with several questions with 5 likert scale points and we would like to compare the answers to those of another likert like question with 10 points. The approach we thought of is a ...
1
vote
0answers
106 views

Using t-test for feature selection after z-scoring data?

Suppose I have a high-dimensional dataset, and a binary classification problem. I want to use the two-sample t-test for feature selection. If the data has been normalized by z-scoring (so it has zero ...
1
vote
0answers
51 views

Why normalized feature weights for linear regression are bad feature importance predictors

I am trying to interpret a linear regression model. I assumed using absolute value of feature weight coefficients as indicators of influence of input variable onto output variable. However, it seems ...
1
vote
0answers
34 views

Feature boosting via rescaling in logistic regression and linear SVMs

If I were expressing a problem in terms of binary features, all encoded as {0,1}, could I boost some features by encoding them as {0,2}? Would the effect change based on whether I used either of the ...