Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
63 views

Random Forests and Feature selection [duplicate]

First, I split my training set into 10 parts. 9 parts of it, I use as LS and the other one as TS. I now want to do feature selection, so I do feature selection on 9 parts. I use Random Forest to do ...
16
votes
4answers
648 views

When wouldn't I use LASSO for model selection?

Assume that you need to build a linear model to make predictions for new observations, and that there is uncertainty about which subset of variables should be included in the model. You are only ...
1
vote
2answers
167 views

CV on training set with feature selection

I've got a problem with CV on feature selection. I've used a method, but I don't know it's correct... I split my data into 70% training set and 30% test set I work now with my training set. I do on ...
0
votes
1answer
166 views

Feature selection and cross validation

I'm working on a project and I would like to know if the following strategy is good/correct. Sorry if this is a basic/stupid idea (I'm new to this). The input is a dataset with 2.500 features and ...
0
votes
0answers
20 views

Comparing 2 different sets of features

So let's say I have two different sets of features A and B. I'm trying to determine which set of features is the best. I'm using leave-one-out cross validation as the final metric as my data set is ...
1
vote
0answers
76 views

How to explain difference of importance between feature selection and model quality?

I have a data collection with a mixed feature set consisting of both numerical features and text features. The number of numerical features is quite small, i.e., 6, comparing to the number of text ...
0
votes
1answer
89 views

Random forest importance differs between rf$importance and importance()

My model is working ok (the AUC is 0.7) but the importances from a randomForest run for my binary classification problem differ depending on how I retrieve them. Is ...
0
votes
1answer
38 views

Predictive features with high presence in one class

I am doing a logistic regression to predict the outcome of a binary variable, say whether a journal paper gets accepted or not. The independent variable or predictors are all the phrases used in these ...
1
vote
1answer
84 views

Numeric example of data for special case of stepwise linear regression

Stepwise Regression works as follows if I'm correct: fit the initial model add the variable which has its f-stat larger than a in-threshold and repeat step 2. if there are no candidates to enter - ...
0
votes
0answers
19 views

Why the maxStages argument in biglars.fit does not work

Why doesn't the biglars.fit function work when maxStages is specified? I've tried multiple values and multiple ways of casting $y$ but it doesn't work. ...
2
votes
3answers
500 views

How to divide feature set for selection and training

I have training data with 260 observations that have a total of 7 classes. Each observation has 120 features. I applied feature selection based on the Bhattacharyya Algorithm and got the top 40 ...
1
vote
1answer
59 views

finding an optimal subgroup of binary indicators

My dependent variable is continuous variable that measures the (potential) success of a person in some activity. I have hundreds of binary indicators, each indicates about the existence of a specific ...
1
vote
0answers
169 views

Cross validation for variable selection and coefficient shrinkage?

Is cross validation an appropriate technique for variable selection and regression coefficient shrinkage? A former colleague of mine used 10-fold CV to compare the regression coefficients from the ...
0
votes
0answers
226 views

Metric warning using caret's rfe

I am using the caret package to do feature selection with rfe while training a knn ...
0
votes
0answers
43 views

Variable selection (automated)

I was wondering whether the following mechanical selection procedure will result in a possible bias. First let me introduce the first procedure, we start with a model and only look at the t-value and ...
8
votes
4answers
188 views

Lasso-ing the order of a lag?

Suppose I have longitudinal data of the form $\mathbf Y = (Y_1, \ldots, Y_J) \sim \mathcal N(\mu, \Sigma)$ (I have multiple observations, this is just the form of a single one). I'm interested in ...
0
votes
0answers
13 views

Sparse variable selection algorithms that account for parameter changes

I am variable selecting for a time-series forecasting model that has parameters sampled from a high variance sampling distribution centred near zero and that undergo changes over time. Each predictor ...
13
votes
2answers
671 views

Why does the Lasso provide Variable Selection?

I've been reading Elements of Statistical Learning, and I would like to know why the Lasso provides variable selection and ridge regression doesn't. Both methods minimize the residual sum of squares ...
3
votes
0answers
177 views

When would I choose Lasso over Elastic Net

What are the scenarios where Lasso is likely to perform better than Elastic Net (out of sample prediction)?
2
votes
3answers
108 views

What if Lasso selects transformed terms but not untransformed terms

Suppose I have standard normal features $X_i \in \{X_i : i \in \{1,...,1000\}\}$. I extend this set of predictors with transformations as follows: $\{X_i,X_i^2,X_iI(X_i > 0) : i \in ...
0
votes
1answer
154 views

Best subset selection

My statistical learning text claims that for best subset selection, 2^p total models must be fit through regression if for p covariates, we fit p choose k models at each k, k = 1,...,p. I interpret ...
0
votes
0answers
30 views

Recursive Feature Elimination Fails to Output as Expected

Currently I am using rfe function in the "caret" package to do feature selection. There are 380 variables as input candidates. I have done many trials and I noticed that something weird always ...
0
votes
0answers
138 views

Find weight of features for feature selection

I have a data set of videos from which I need to recognize the emotion of the speaker. For that reason I have some markers on the face of the speaker. I detect their movement as the speaker speaks and ...
2
votes
2answers
202 views

Variable selection with groups of predictors that are highly correlated

What variable selection approach should I consider if I have thousands of predictors with clusters that are extremely correlated? For example I might have a predictor set $X:= ...
1
vote
1answer
103 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
1
vote
1answer
355 views

Decision trees variable (feature) scaling and variable (feature) normalization (tuning) required in which implementations?

In many machine learning algorithms, feature scaling (aka variable scaling, normalization) is a common prepocessing step Wikipedia - Feature Scaling -- this question was close Question#41704 - How and ...
0
votes
4answers
137 views

Machine learning input relationships

After learning about a few machine-learning models (NN, SVM, decision trees), I was wondering if these models are able to find inherent relationships when learning. For example, if I feed it two ...
0
votes
1answer
135 views

How Can I use some variables selected by LASSO?

I am very new about statistics. So, please understand if my question is somewhat awkward, and please give me related any advice. I have some data set. X = 500 x 100 (500 observations x 100 ...
2
votes
2answers
250 views

Variable selection in time-series forecasting

I have a time-series forecasting task and would like some input on variable selection and regularisation. My problem has the following characteristics: 2,000,000 sample size. Most of the time, no ...
5
votes
3answers
1k views

Selecting the best subset of variables for parsimonious binary logistic regression models

In addition to PROC VARCLUS, randomForest, glmnet, and assessing multicollinearity among potential predictor variables (without regards to the outcome of interest), I am seeking other methods of ...
2
votes
1answer
69 views

Distinguishing two datasets

I have two datasets from some Web store (like Amazon). Datasets have one and the same structure. Each record in these datasets has the following attributes: ...
1
vote
1answer
36 views

Interpretation of feature selection task

So I am given the following question Data set sample5.txt has a 20-dimensional input $x$ in $\mathbb{R}^{20}$ but we suspect that many of these are actually irrelevant. Could you model the ...
1
vote
1answer
94 views

k-fold feature selection

I have a data set with 20 K variables. I have tried to select some features via Boruta and FSelector but I could not achieve ...
3
votes
0answers
96 views

Variable reduction by means of ANOVA?

I have a typical problem with several variables and a large amount of data which are not important right now. The goal of the study is to relate variable $Y$ with variables $X_1,X_2,...,X_n$. I have ...
3
votes
0answers
87 views

Mutual information/pointwise mutual information for measuring prediction

I want to measure how well I predict a vector $Y$ (vector not a label) for observation $X$. Both $X$ and $Y$ have the same set of features ($1\times n$). For that, I thought of "scoring" the ...
2
votes
0answers
75 views

Confusion related to feature selection

Well my objective is to predict solar energy radiation at a particular location given some features like wind, temperature, humidity ... I have a total data for 10 years where I have the measurement ...
-1
votes
1answer
199 views

Implement Forward, Backward, Step and LASSO in VB .NET

My client wants me to implement Variable selection methods i.e. Forward, Backward, Step and LASSO in VB .Net platform including p-value and AIC. I have no idea about the steps involved to calculate ...
1
vote
1answer
48 views

How to deal with features only available for a few instances?

I am working with a regression problem. I have some features which are only available for a few instances. But with those few instances based on those features we can build a model which gives ...
1
vote
2answers
991 views

Random Forest: IncNodePurity and Feature Selection for Binary Logistic Regression

After creating a Random Forest object using randomForest with around 500 candidate variables, I used importance(object) to ...
2
votes
1answer
203 views

Combining Exploratory Factor Analysis with Random Forest for Binary Logistic Regression Feature Selection

For those of you familiar with Exploratory Factor Analysis (EFA) and Random Forest (RF), I have recently had an idea of combining these two methods to reduce the number of potential predictor ...
7
votes
2answers
3k views

Feature selection with Random Forests

I have a dataset with mostly financial variables (120 features, 4k examples) which are mostly highly correlated and very noisy (technical indicators, for example) so I would like to select about max ...
0
votes
1answer
316 views

Exploratory Factor Analysis for Binary Logistic Regression Variable Selection

I have a great interest in learning new methods(at least to me) of variable selection in regards to binary logistic regression when I am working with over 500 potential predictor variables and have ...
1
vote
0answers
77 views

Feature selection using correlation

I am trying to do some feature selection using correlation. However, I found that my features are not that correlated. The highest correlation was 0.08. So I am not sure if this is a useful thing to ...
1
vote
0answers
148 views

Features selection by filter methods for multivariate time series

I have a data set in which the samples are multivariate (about 30 variable/features) time series. These samples refer to two classes. I would like to select the variables more relevant to discriminate ...
3
votes
1answer
289 views

Using Mutual Information for Binary Logistic Regression Variable Selection

In addition to proc varclus, randomForest, and assessing multicollinearity among potential predictor variables, I am seeking ...
1
vote
1answer
75 views

Can I select a subset of predictor variables without using step()?

I have a set of 20 predictor variables, and I want to formulate a regression model by applying my own variable selection technique basically with backward approach (just for an experiment purpose.) ...
2
votes
0answers
69 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
2
votes
0answers
31 views

Correct order of performing imputation and variable selection

This is a general question about performing data analysis. I have a data set with ~1000 sample size and 200 features. Some of features have more than 50% missing or even higher. The missing pattern is ...
5
votes
1answer
164 views

A Bayesian perspective on omitted-variable bias (and other covariate-selection bias problems)

As I know OVB, from a frequentist education, when you leave a variable $(z)$ out of your control set $(X)$ that is correlated with both your independent variable of interest (treatment $T$) and your ...
2
votes
0answers
271 views

Feature importance scores of SVM multiclass one-vs-one design

Info about dataset: 5 classes, 200 trials, 100 features. (I know about the trial to feature ratio being very low, but can not avoid this here and still got well enough classification results.) ...