Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

-1
votes
0answers
8 views

Alternative to self-organizing maps for image clustering

I'm thinking to use self-organizing maps (with TensorFlow) to group images of people by some characteristics (like the color of their clothes). Any experiences with SOM on such task or do you suggest ...
0
votes
1answer
22 views

How can I find the field which most affects or contributes to decision making in a machine learning algorithm?

Consider the example below. On a larger dataset, it would be fairly obvious that name and gender are not a good indicator of whether a person is an adult or a kid, and that it's age which best decides ...
13
votes
5answers
240 views

Variable selection for predictive modeling really needed in 2016?

This question has been asked on CV some yrs ago, it seems worth a repost in light of 1) order of magnitude better computing technology (e.g. parallel computing, HPC etc) and 2) newer techniques, e.g. ...
0
votes
0answers
9 views

Identifying sequential patterns and deciding which ones are useful

So, basically I have a problem in which I have, over time, the appearance of different features, each feature containing different categories (where categories belonging to the same feature cannot ...
0
votes
0answers
18 views

log loss and squared loss in shrinkage tuning in R?

My model is logistic regression. Is there a way to tune the parameter lambda of lasso or ridge based on cross-validated log-loss and brier(eg. proper scores?) in any R packages? I'm using glmnet ...
0
votes
0answers
8 views

When to use Group lasso over lasso?

Two cases: When should numerous numerical predictors be grouped? is it just based off some theoretical knowledge on the predictors? When should levels(>2) in a factor be grouped together?
0
votes
0answers
12 views

Feature selection for an ordered logit model (R)

I'm using an ordered logit model to predict credit ratings/risk (1-8, ordinal) as a function of 126 predictor variables. (See: https://www.kaggle.com/c/prudential-life-insurance-assessment/data for ...
0
votes
0answers
7 views

Can different data mining algorithms cross check each other's feature selection?

I have worked with the same data set for a little while, using a number of different data mining algorithms. As a result, I have developed a short list of predators which are virtually always useful - ...
0
votes
1answer
32 views

Feature selection: PCA vs intuition? [on hold]

Which one should I choose? How can I combine them (i.e. in series or parallel)? What if there are dummy features in my data? What if my intuition messes things up?
0
votes
0answers
14 views

Cross validated loglikelihood?

This is probably a silly question: I was playing around with penalized package and cvl outputs a cross validated loglikelihood and another measure just called loglikelihood which is suppose to be ...
1
vote
1answer
32 views

Results from rfe function (caret) to compute average metrics - R

I am computing a SVM-RFE model with the rfe function of the caret package, but I am a bit confused about the results. My code ...
0
votes
1answer
73 views

More features than data points in linear regression

In a dataset with more features (e.g. 120) than data points (e.g. 60) what are the techniques commonly used to select the best features to apply linear regression? Obviously there is an efficiency ...
1
vote
0answers
35 views

Why use group lasso instead of lasso?

I have read the that the group lasso is used for variable selection and sparsity in a group of variables. I want to know the intuition behind this claim. Why is group lasso preferred to lasso? Why ...
-3
votes
1answer
40 views

Why should I choose features or plot them manually when there are built-in functions to do that?

Why should I select variables due to my intuition if there are builtin functions in sklearn python like SelectKBest() and PCA() If I plot graphs of features of the data to see if they can detect the ...
0
votes
0answers
3 views

how to extract selectKbest feature scores from pipeline? [migrated]

My code is below. I'm wondering how to extract the selectKbest .scores_ from my pipeline, after the classifier has been trained. ...
1
vote
1answer
30 views

General-to-specific subset selection (“Autometrics”) performing well in macroeconomics

I wonder why general-to-specific (GETS) subset selection and particularly the Autometrics algorithm are performing well in macroeconomic modelling/forecasting. How does Autometrics work? Doornik ...
0
votes
2answers
34 views

Is it possible to make the non-separable data more separable by any methods of feature selection, extraction or transformation?

Could these data (in the figure below) be separated by any means of feature extraction, transformation, or it's just a waste of time to make the three classes separable if they "in fact" weren't ...
0
votes
0answers
5 views

Corelations attributes with decision - multi class

I have three integer decision classes: 1,2,3. Tell me please, if it make sense to compute corelations attributes with decision, and given it select ...
0
votes
1answer
22 views

How to make this data in the following figure separable for the classification into three classes?

The figure below shows the PCA projections of inputs which are 14 meteorological features, (i.e. wind, temperature, humidity, pressure, and so on.) I would like to use any technique to make it more ...
0
votes
0answers
20 views

Fisher Discriminant Analysis vs ANOVA [closed]

Both FDA and ANOVA talks about minimizing within variance and maximizing across across variance. Lets say there are 3 classes for which feature f1 data is available. We can apply both of the above ...
2
votes
1answer
56 views

Lasso will not remove correlated variables

The very essence of lasso is that it is supposed to select only one of two correlated variables. However, when I include two highly correlated predictors (they are correlated with each other at level ...
1
vote
0answers
21 views

Feature ranking for *known* clusters

I am aware of feature ranking (i.e. selecting the 'best' features) for a binary classification task based on some model, however, I was wondering how to do this in the absence of a model? For example, ...
0
votes
1answer
24 views

How to get the best out of a “bad” set of features for regression?

I'm trying to learn a regression model for a computer vision / pattern recognition task, where I try to estimate a continuous variable from a set of visual features. I have done preliminary ...
1
vote
0answers
31 views

Double lasso variable selection

Currently I am learning about variable selection and lasso. I found the paper by Urminsky et al. "Using Double-Lasso Regression for Principled Variable Selection" (2016) which proposes a double lasso ...
0
votes
1answer
30 views

Can cluster analysis of preclassified items gives idea about the classification performance?

Suppose in a classification we have a dataset with many features and their class, we want to select some features using which we can construct a classifier. We perform the cluster evaluation for the ...
0
votes
1answer
29 views

Logistic regression categorical variable interpretation after transformed into dummy variable

Before training a glm model (in R), predictors were transformed into matrix and highly correlated/near zero variance variables were excluded: ...
1
vote
1answer
12 views

Method for variable shortlisting (where sample size is small and variable number is large)

I have 10 different herb samples, and each sample has ~400 chemical components in varying amounts (all numerical variables). I would like to determine which of these variables contribute to the ...
0
votes
1answer
22 views

General Criteria for Feature Quality

It is generally accepted that the most important factor for successful machine learning is quality feature engineering: Feature Engineering is the Key At the end of the day, some machine ...
0
votes
0answers
9 views

How to utilize the features available in the train data but not in the test data to train the algorithm?

I have recently worked on some kaggle competitions and found some of them provided some features only for the training data but not for the test. For example this Expedia one does not have session ...
0
votes
1answer
31 views

Use Lasso Logistic Regression to Analyze Binary Data with

I am involved with a medical research that analyzes Coronary Artery Disease. The dataset has a couple of predictors such as age, gender, race, certain symptons and medical standard procedures to be ...
1
vote
1answer
47 views

Can I apply OLS (multiple regression) to panel data to identify significant variables?

I have panel data for a 5-year period and want to explore the determinants of car prices (number of doors, house power, etc.). Is it appropriate to use OLS or multiple regression to explore the ...
0
votes
0answers
9 views

Should one drop features not returned by xgb.importance?

The xgb.importance() function within R package xgboost was used. Have some questions in mind, and cannot seem to find a direct answer somewhere else, if someone can address these: When ...
0
votes
0answers
20 views

Feature selection: wrapper or embedded?

I have searched for information on what could be the reasons for preferring wrapper feature selection to embedded feature selection. I found a comparative study which tells when filters are better, ...
2
votes
1answer
77 views

What can be the reason to do feature selection based on variance before doing PCA?

I have noticed that when applying PCA to large datasets, people often will first subset the data considerably. Sometimes people just randomly take a subset of the features/variables, but often they ...
4
votes
1answer
93 views

Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization. My question: do people normally use ...
1
vote
1answer
64 views

what is “principled feature selection”?

i see the expression "principled feature selection" in titles of various Machine Learning papers and generally in the literature but nowhere do authors really define what they mean. "principled" as ...
0
votes
2answers
30 views

How to combine a set of features and report their effectiveness with only one number?

I have a binary classification problem. My data set has 100 features with 10 different categories (10 features per category). I want to report the effectiveness (in terms of classification) with a ...
0
votes
0answers
24 views

Random Forest: Strange Feature Importance Results for Constant Variables

I've been using the RandomForestClassifier in python's Sklearn package to assess the importance of the features in a large dataset with features that are both binary and continuous. I've done quite a ...
0
votes
0answers
21 views

Interpreting results from lasso regression?

I have a time series data set with about 2million observations and 31 variables, which I break to a few thousand using threshold value for my dependent variable. I am using lasso regression in R to ...
1
vote
1answer
37 views

How to determine which variables to be used for cluster analysis

I have about 10 variables (features) and want to do cluster analysis of cases (data points). I have a number of ideas about which variables to be included for cluster analysis: Plot the variables ...
0
votes
0answers
25 views

Locality Sentive Hashing for Dimentionality Reduction or Feature clustering

So I have read up on LSH and Asymmetric hashing as proposed by Google for their google correlate algorithm. These work by only comparing similar items due to the multiple random hashes, however we are ...
0
votes
0answers
8 views

Fixed-effects variable selection for mixed-effects regression

Does anybody know if it is possible to apply some "feature selection" algorithm to a dataset prior to creating a mixed-effects regression model? I am trying to implement such a modelling in Matlab, ...
0
votes
1answer
12 views

taking average of several models and feature sets

just a quick question that i cant seem to find a definitive answer for. When im doing feature selection, i end up with a list of the top performing sets. Would it make sense to use the top 10 sets ...
0
votes
0answers
41 views

Caret classification: feature selection & unbalanced data

I have a two-class classification problem with very unbalanced data (~1:1000 Yes/No ratio). The initial model class I'd like to try is regular glm. So there are two issues need to be addressed: 1) ...
0
votes
0answers
11 views

How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...
0
votes
0answers
13 views

Correct Feature Selection Methodology?

I am running a weighted multiple linear regression where my independent variables take binary values, 0 and 1. The dependent variable y, takes numeric values (positive as well as negative). The ...
0
votes
1answer
25 views

Comparing and evaluating win probabilities in sports from different settings

Background I'm trying to predict the probability that the home teams wins a certain sports game, for each minute of the game. Taking these win probabilities together produces a nice visual of the ...
0
votes
0answers
21 views

T-test for feature selection

I want to reduce features (voxels) in my fMRI data. I applied t-test between two conditions and select only those features which are significant (p<0.05). After that I divided the data into ...
0
votes
1answer
42 views

Main Drawbacks of stepwise regression

People typically prefer the Lasso or other methods to stepwise regression. What are the main problems in stepwise regression which makes it unreliable specifically the problems with forward selection ...
0
votes
0answers
9 views

How does Weka chiSquaredAttributeEval generates single attribute selection list while Chi Square itself is class based?

I have implemented my own Chi-Square ranker in C# however the example i found on the internet shows that Chi-Square ranks the each attribute within its class However Weka generates attributes as a ...