Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
0answers
27 views

Comparing and ranking differentiating attributes across groups

I'm looking for some help on how to approach this problem. Say I have two or more groups of people. Each group has characteristics and attributes. For example, say we have the following two groups: ...
0
votes
1answer
79 views

Using non-significant variables in model

I am trying to build a credit scoring model and have discovered and interesting approach for feature selection. I am looping through all features and removing them one by one (using variable ...
0
votes
1answer
29 views

In feature selection, are there any rules on choosing metrics to mesure relevance? (MI / Fisher score / correlation coefficient, etc)

This is a rather general question. If the question is vague and hard to answer in a few lines, I'd be happy if someone just point me to some readings. Thanks in Advance. I am working on a multi-class ...
1
vote
1answer
71 views

Does PCA do something else apart from selecting features with the most variance?

While experimenting with Spark library MlLib, I questioned myself if I understood well the mechanism of PCA algorithm, because output of MlLib algorithm was not what I expected to get. so for given ...
3
votes
1answer
25 views

Extension to SAFE screening rule for Lasso

In El Ghaoui et al. (2010), "Safe feature elimination in sparse learning" and following works, screening rules are derived for Lasso (as well as other L1-penalized problems): $ \min_w \|y-X w\|^2 + \...
0
votes
2answers
30 views

choosing a model after feature selection process

so ive been selecting features for a regression problem and have obtained a list of the best performing feature sets. (note my list is actually several thousand lines long) 188.493 186.989 [379.45, 0....
0
votes
1answer
28 views

number of features in feature selection for text mining problems

Let's say for a text mining problem (e.g creating a predictive model using text analysis), using a feature selection method (e.g TF-IDF) we come up with 1000 features/words/tokens. Is there some ...
1
vote
0answers
20 views

Pedagogical example of feature selection for model building

I am looking for a good pedagogical example use of feature selection for model building. The purpose is to expose students to some very basic methods for feature selection, in the context of boolean ...
1
vote
1answer
29 views

Study design using multinomial vs logistic regression?

Suppose that I have a categorical response variable that consists of group 1-3, and I hope to see if predictors can differentiate group 1 vs group 3 (group 2 not included). The response variable is ...
0
votes
2answers
40 views

How can I find the field which most affects or contributes to decision making in a machine learning algorithm?

Consider the example below. On a larger dataset, it would be fairly obvious that name and gender are not a good indicator of whether a person is an adult or a kid, and that it's age which best decides ...
34
votes
6answers
1k views

Variable selection for predictive modeling really needed in 2016?

This question has been asked on CV some yrs ago, it seems worth a repost in light of 1) order of magnitude better computing technology (e.g. parallel computing, HPC etc) and 2) newer techniques, e.g. [...
0
votes
0answers
27 views

Identifying sequential patterns and deciding which ones are useful

So, basically I have a problem in which I have, over time, the appearance of different features, each feature containing different categories (where categories belonging to the same feature cannot ...
0
votes
0answers
21 views

log loss and squared loss in shrinkage tuning in R?

My model is logistic regression. Is there a way to tune the parameter lambda of lasso or ridge based on cross-validated log-loss and brier(eg. proper scores?) in any R packages? I'm using glmnet ...
0
votes
0answers
11 views

When to use Group lasso over lasso?

Two cases: When should numerous numerical predictors be grouped? is it just based off some theoretical knowledge on the predictors? When should levels(>2) in a factor be grouped together?
0
votes
0answers
13 views

Feature selection for an ordered logit model (R)

I'm using an ordered logit model to predict credit ratings/risk (1-8, ordinal) as a function of 126 predictor variables. (See: https://www.kaggle.com/c/prudential-life-insurance-assessment/data for ...
0
votes
0answers
12 views

Can different data mining algorithms cross check each other's feature selection?

I have worked with the same data set for a little while, using a number of different data mining algorithms. As a result, I have developed a short list of predators which are virtually always useful - ...
0
votes
0answers
15 views

Cross validated loglikelihood?

This is probably a silly question: I was playing around with penalized package and cvl outputs a cross validated loglikelihood and another measure just called loglikelihood which is suppose to be "...
1
vote
1answer
64 views

Results from rfe function (caret) to compute average metrics - R

I am computing a SVM-RFE model with the rfe function of the caret package, but I am a bit confused about the results. My code is:...
0
votes
1answer
85 views

More features than data points in linear regression

In a dataset with more features (e.g. 120) than data points (e.g. 60) what are the techniques commonly used to select the best features to apply linear regression? Obviously there is an efficiency ...
1
vote
0answers
45 views

Why use group lasso instead of lasso?

I have read the that the group lasso is used for variable selection and sparsity in a group of variables. I want to know the intuition behind this claim. Why is group lasso preferred to lasso? Why ...
-3
votes
1answer
44 views

Why should I choose features or plot them manually when there are built-in functions to do that?

Why should I select variables due to my intuition if there are builtin functions in sklearn python like SelectKBest() and PCA() If I plot graphs of features of the data to see if they can detect the ...
1
vote
1answer
39 views

General-to-specific subset selection (“Autometrics”) performing well in macroeconomics

I wonder why general-to-specific (GETS) subset selection and particularly the Autometrics algorithm are performing well in macroeconomic modelling/forecasting. How does Autometrics work? Doornik "...
0
votes
2answers
51 views

Is it possible to make the non-separable data more separable by any methods of feature selection, extraction or transformation?

Could these data (in the figure below) be separated by any means of feature extraction, transformation, or it's just a waste of time to make the three classes separable if they "in fact" weren't ...
0
votes
0answers
5 views

Corelations attributes with decision - multi class

I have three integer decision classes: 1,2,3. Tell me please, if it make sense to compute corelations attributes with decision, and given it select ...
0
votes
1answer
32 views

How to make this data in the following figure separable for the classification into three classes?

The figure below shows the PCA projections of inputs which are 14 meteorological features, (i.e. wind, temperature, humidity, pressure, and so on.) I would like to use any technique to make it more ...
3
votes
1answer
116 views

What to do when lasso does not remove correlated variables?

The very essence of lasso is that it is supposed to select only one of two correlated variables. However, when I include two highly correlated predictors (they are correlated with each other at level ...
1
vote
0answers
22 views

Feature ranking for *known* clusters

I am aware of feature ranking (i.e. selecting the 'best' features) for a binary classification task based on some model, however, I was wondering how to do this in the absence of a model? For example, ...
0
votes
1answer
33 views

How to get the best out of a “bad” set of features for regression?

I'm trying to learn a regression model for a computer vision / pattern recognition task, where I try to estimate a continuous variable from a set of visual features. I have done preliminary ...
1
vote
0answers
48 views

Double lasso variable selection

Currently I am learning about variable selection and lasso. I found the paper by Urminsky et al. "Using Double-Lasso Regression for Principled Variable Selection" (2016) which proposes a double lasso ...
0
votes
1answer
37 views

Can cluster analysis of preclassified items gives idea about the classification performance?

Suppose in a classification we have a dataset with many features and their class, we want to select some features using which we can construct a classifier. We perform the cluster evaluation for the ...
0
votes
1answer
34 views

Logistic regression categorical variable interpretation after transformed into dummy variable

Before training a glm model (in R), predictors were transformed into matrix and highly correlated/near zero variance variables were excluded: ...
1
vote
1answer
15 views

Method for variable shortlisting (where sample size is small and variable number is large)

I have 10 different herb samples, and each sample has ~400 chemical components in varying amounts (all numerical variables). I would like to determine which of these variables contribute to the ...
0
votes
1answer
28 views

General Criteria for Feature Quality

It is generally accepted that the most important factor for successful machine learning is quality feature engineering: Feature Engineering is the Key At the end of the day, some machine ...
0
votes
0answers
12 views

How to utilize the features available in the train data but not in the test data to train the algorithm?

I have recently worked on some kaggle competitions and found some of them provided some features only for the training data but not for the test. For example this Expedia one does not have session ...
0
votes
1answer
49 views

Use Lasso Logistic Regression to Analyze Binary Data with

I am involved with a medical research that analyzes Coronary Artery Disease. The dataset has a couple of predictors such as age, gender, race, certain symptons and medical standard procedures to be ...
1
vote
1answer
53 views

Can I apply OLS (multiple regression) to panel data to identify significant variables?

I have panel data for a 5-year period and want to explore the determinants of car prices (number of doors, house power, etc.). Is it appropriate to use OLS or multiple regression to explore the ...
0
votes
0answers
15 views

Should one drop features not returned by xgb.importance?

The xgb.importance() function within R package xgboost was used. Have some questions in mind, and cannot seem to find a direct answer somewhere else, if someone can address these: When xgb....
0
votes
0answers
32 views

Feature selection: wrapper or embedded?

I have searched for information on what could be the reasons for preferring wrapper feature selection to embedded feature selection. I found a comparative study which tells when filters are better, ...
2
votes
1answer
89 views

What can be the reason to do feature selection based on variance before doing PCA?

I have noticed that when applying PCA to large datasets, people often will first subset the data considerably. Sometimes people just randomly take a subset of the features/variables, but often they ...
4
votes
1answer
127 views

Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization. My question: do people normally use ...
1
vote
1answer
69 views

what is “principled feature selection”?

i see the expression "principled feature selection" in titles of various Machine Learning papers and generally in the literature but nowhere do authors really define what they mean. "principled" as ...
0
votes
2answers
31 views

How to combine a set of features and report their effectiveness with only one number?

I have a binary classification problem. My data set has 100 features with 10 different categories (10 features per category). I want to report the effectiveness (in terms of classification) with a ...
0
votes
0answers
41 views

Random Forest: Strange Feature Importance Results for Constant Variables

I've been using the RandomForestClassifier in python's Sklearn package to assess the importance of the features in a large dataset with features that are both binary and continuous. I've done quite a ...
0
votes
0answers
26 views

Interpreting results from lasso regression?

I have a time series data set with about 2million observations and 31 variables, which I break to a few thousand using threshold value for my dependent variable. I am using lasso regression in R to ...
1
vote
1answer
43 views

How to determine which variables to be used for cluster analysis

I have about 10 variables (features) and want to do cluster analysis of cases (data points). I have a number of ideas about which variables to be included for cluster analysis: Plot the variables ...
0
votes
0answers
30 views

Locality Sentive Hashing for Dimentionality Reduction or Feature clustering

So I have read up on LSH and Asymmetric hashing as proposed by Google for their google correlate algorithm. These work by only comparing similar items due to the multiple random hashes, however we are ...
0
votes
0answers
13 views

Fixed-effects variable selection for mixed-effects regression

Does anybody know if it is possible to apply some "feature selection" algorithm to a dataset prior to creating a mixed-effects regression model? I am trying to implement such a modelling in Matlab, ...
0
votes
1answer
18 views

taking average of several models and feature sets

just a quick question that i cant seem to find a definitive answer for. When im doing feature selection, i end up with a list of the top performing sets. Would it make sense to use the top 10 sets ...
0
votes
0answers
63 views

Caret classification: feature selection & unbalanced data

I have a two-class classification problem with very unbalanced data (~1:1000 Yes/No ratio). The initial model class I'd like to try is regular glm. So there are two issues need to be addressed: 1) ...
0
votes
0answers
13 views

How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...