Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
1answer
234 views

Decision trees variable (feature) scaling and variable (feature) normalization (tuning) required in which implementations?

In many machine learning algorithms, feature scaling (aka variable scaling, normalization) is a common prepocessing step Wikipedia - Feature Scaling -- this question was close Question#41704 - How and ...
0
votes
0answers
31 views

Confusion related to feature engineering

I was reading this tutorial where they mentioned ...
0
votes
4answers
132 views

Machine learning input relationships

After learning about a few machine-learning models (NN, SVM, decision trees), I was wondering if these models are able to find inherent relationships when learning. For example, if I feed it two ...
0
votes
1answer
126 views

How Can I use some variables selected by LASSO?

I am very new about statistics. So, please understand if my question is somewhat awkward, and please give me related any advice. I have some data set. X = 500 x 100 (500 observations x 100 ...
2
votes
2answers
211 views

Variable selection in time-series forecasting

I have a time-series forecasting task and would like some input on variable selection and regularisation. My problem has the following characteristics: 2,000,000 sample size. Most of the time, no ...
5
votes
3answers
838 views

Selecting the best subset of variables for parsimonious binary logistic regression models

In addition to PROC VARCLUS, randomForest, glmnet, and assessing multicollinearity among potential predictor variables (without regards to the outcome of interest), I am seeking other methods of ...
0
votes
0answers
46 views

Network/structure learning

Given a data set $\mathbf{X}\in\mathbb{R}^{n\times p}$, where $n$ is the number of samples (observations) and $p$ is the number of features, I would like to know what kind of methods exist for ...
2
votes
1answer
67 views

Distinguishing two datasets

I have two datasets from some Web store (like Amazon). Datasets have one and the same structure. Each record in these datasets has the following attributes: ...
1
vote
1answer
36 views

Interpretation of feature selection task

So I am given the following question Data set sample5.txt has a 20-dimensional input $x$ in $\mathbb{R}^{20}$ but we suspect that many of these are actually irrelevant. Could you model the ...
1
vote
1answer
90 views

k-fold feature selection

I have a data set with 20 K variables. I have tried to select some features via Boruta and FSelector but I could not achieve ...
0
votes
0answers
29 views

Connection between feature selection and hypothesis testing

I have a dataset for cell-phone accounts and I am trying to predict whether or not an account will cancel given some input features. One such feature is the number of devices an account owns. I am ...
3
votes
0answers
91 views

Variable reduction by means of ANOVA?

I have a typical problem with several variables and a large amount of data which are not important right now. The goal of the study is to relate variable $Y$ with variables $X_1,X_2,...,X_n$. I have ...
0
votes
0answers
69 views

LASSO method: prediction for multi-dimentional reponses

I have a feature matrix, that is 'X' 2000 (observation) x 200 (variable). I also have a response matrix, that is 'Y' 2000 (response) x 2 (variable). I would like to apply LASSO method to the data ...
3
votes
0answers
80 views

Mutual information/pointwise mutual information for measuring prediction

I want to measure how well I predict a vector $Y$ (vector not a label) for observation $X$. Both $X$ and $Y$ have the same set of features ($1\times n$). For that, I thought of "scoring" the ...
0
votes
0answers
119 views

Time series classification

I am classifying a set of time series inputs after creating independent features from every $n$ samples and running machine learning algorithms. I get good accuracy based on many error metrics on the ...
2
votes
0answers
70 views

Confusion related to feature selection

Well my objective is to predict solar energy radiation at a particular location given some features like wind, temperature, humidity ... I have a total data for 10 years where I have the measurement ...
-1
votes
1answer
192 views

Implement Forward, Backward, Step and LASSO in VB .NET

My client wants me to implement Variable selection methods i.e. Forward, Backward, Step and LASSO in VB .Net platform including p-value and AIC. I have no idea about the steps involved to calculate ...
1
vote
1answer
47 views

How to deal with features only available for a few instances?

I am working with a regression problem. I have some features which are only available for a few instances. But with those few instances based on those features we can build a model which gives ...
0
votes
2answers
824 views

Random Forest: IncNodePurity and Feature Selection for Binary Logistic Regression

After creating a Random Forest object using randomForest with around 500 candidate variables, I used importance(object) to ...
2
votes
1answer
187 views

Combining Exploratory Factor Analysis with Random Forest for Binary Logistic Regression Feature Selection

For those of you familiar with Exploratory Factor Analysis (EFA) and Random Forest (RF), I have recently had an idea of combining these two methods to reduce the number of potential predictor ...
7
votes
2answers
2k views

Feature selection with Random Forests

I have a dataset with mostly financial variables (120 features, 4k examples) which are mostly highly correlated and very noisy (technical indicators, for example) so I would like to select about max ...
0
votes
1answer
272 views

Exploratory Factor Analysis for Binary Logistic Regression Variable Selection

I have a great interest in learning new methods(at least to me) of variable selection in regards to binary logistic regression when I am working with over 500 potential predictor variables and have ...
1
vote
0answers
74 views

Feature selection using correlation

I am trying to do some feature selection using correlation. However, I found that my features are not that correlated. The highest correlation was 0.08. So I am not sure if this is a useful thing to ...
1
vote
0answers
138 views

Features selection by filter methods for multivariate time series

I have a data set in which the samples are multivariate (about 30 variable/features) time series. These samples refer to two classes. I would like to select the variables more relevant to discriminate ...
3
votes
1answer
261 views

Using Mutual Information for Binary Logistic Regression Variable Selection

In addition to proc varclus, randomForest, and assessing multicollinearity among potential predictor variables, I am seeking ...
1
vote
1answer
71 views

Can I select a subset of predictor variables without using step()?

I have a set of 20 predictor variables, and I want to formulate a regression model by applying my own variable selection technique basically with backward approach (just for an experiment purpose.) ...
2
votes
0answers
65 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
2
votes
0answers
31 views

Correct order of performing imputation and variable selection

This is a general question about performing data analysis. I have a data set with ~1000 sample size and 200 features. Some of features have more than 50% missing or even higher. The missing pattern is ...
5
votes
1answer
150 views

A Bayesian perspective on omitted-variable bias (and other covariate-selection bias problems)

As I know OVB, from a frequentist education, when you leave a variable $(z)$ out of your control set $(X)$ that is correlated with both your independent variable of interest (treatment $T$) and your ...
2
votes
0answers
254 views

Feature importance scores of SVM multiclass one-vs-one design

Info about dataset: 5 classes, 200 trials, 100 features. (I know about the trial to feature ratio being very low, but can not avoid this here and still got well enough classification results.) ...
4
votes
2answers
334 views

How to handle high dimensional feature vector in probability graph model?

I was doing some NLP related stuff which involves training a hidden Markov model, and use the model to segment sentences. For every sentence, I translate the tokens into feature vectors. The features ...
3
votes
0answers
195 views

Microarray data: suggestions on Feature selection + Model training scheme?

I have a microarray expression dataset (46 samples, thousands of attributes) and I want to perform feature selection first, and, based on this subset of features (shouldn't be more than 4 or 5, based ...
4
votes
1answer
58 views

Which data for feature selection to get unbiased result?

I have a 70 / 30 ratio for train / test data. I have a relatively small feature set (6 features), however, I still want to do feature selection to get rid of any redundant features (I'm guessing 1 of ...
2
votes
1answer
115 views

Number of samples vs Number of features

I've got a set of two classes with 4000 observations total. I've a set of 63 features to construct a predictor. My question is, is there a relation that would prevent overfiting for having too much ...
3
votes
2answers
217 views

Should feature selection be performed only on training data (or all data)?

Should be feature selection performed only on training data (or all data)? I went through some discussions and papers such as Guyon (2003) and Singhi and Liu (2006), but still not sure about right ...
3
votes
1answer
166 views

Feature selection in the training set

I have a classifier, and I am using leave one out cross-validation to assess its performance. On each iteration, I divide the dataset into training and testing sets. The testing set is just the ...
3
votes
1answer
106 views

Select best distance for feature selection

Suppose I have matrix $X \in R^{n \times m}$, where $n$ is the number of individuals and $m$ is the number of features and $X[i,j] \in \{0,1\}$; $1$ indicates that the individual $i$ has the feature ...
1
vote
0answers
154 views

Event Prediction through Machine Learning

I have a large data set consisting of ca. 40 categorical data items and a few interval data items (real numbers, less than 5 such items). Most categories should have a lot of values that repeat ...
0
votes
3answers
277 views

Feature selection before SVM

I have a simple but difficult question. Does feature selection before SVM help? I have a data set that has ~1100 features but a lot of these are redundant data / uncorrelated data. Can someone give me ...
2
votes
1answer
211 views

Stepwise versus L2 regularized logistic regression: dataset-specific performance

I have two data sets from different collections. The second data set is smaller. They were both analyzed with the same methods in order to derive feature sets of 10-30 features each. Each feature set ...
1
vote
1answer
124 views

How to check the features which are selected by LASSO

I am using LASSO (glmnet) to do feature selection. However, how can I check which features are selected?
0
votes
1answer
226 views

Effect of features that are highly correlated with each other on a decision tree

I have a dataset of roughly 500 features and am training a binary classifier using GBM - gradient boosted machines, an ensemble of decision trees. Of these 500 variables, I am sure some are highly ...
0
votes
1answer
187 views

How to model a multi-dimensional feature set for classification

I am new to statistical modelling and so please pardon if the question appears trivial. I have a set of multi-dimensional data ($T$) where each dimension represents features ($f_i$) obtained from a ...
0
votes
1answer
82 views

Regression - Dealing with Correlated, Zero-Sum Predictors

I'm currently working on a regression problem where a subset of the predictor variables are zero-sum. By zero-sum I don't mean they all sum to zero, I simply mean that increasing one implies a ...
5
votes
2answers
2k views

Significance of categorical predictor in logistic regression

I am having trouble interpreting the z values for categorical variables in logistic regression. In the example below I have a categorical variable with 3 classes and according to the z value, CLASS2 ...
1
vote
1answer
403 views

Not all Features Selected by GLMNET Considered Signficant by GLM (Logistic Regression)

I wanted to create a predictive model of mortality after patients had undergone a surgical procedure. But I also wanted to avoid doing what most researchers do by first performing univariate analysis ...
1
vote
1answer
103 views

Dependent variables in regression

I have two variables in a regression problem, where I predict the end interest rate for a loan application. x1: risk band (A+,A,B,C) x2: initial rate (6%, 7%, 8%, 9%) Based on these two variables, ...
1
vote
0answers
31 views

How to use reservoir states for readout and training?

I’m trying to make a Liquid State Machine, I have a spiking neural network as the liquid, and a feedforward neural network that should learn to map the reservoir’s states to the output. I’ve read ...
0
votes
1answer
103 views

How to find important features in this problem

I was just thinking how ML techniques can be applied in the retail industry. Suppose we have data from a retailer who deals with apparel and cloth in this format and for each item there are ...
0
votes
1answer
345 views

What is a good Gini decrease cutoff for feature inclusion based upon random forests?

I am using random forests to try and determine variable importance as part of feature selection for a model I'm working on, and while I can get ranked variable importance by mean decrease in Gini from ...