Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

-1
votes
1answer
30 views

Why should I choose features or plot them manually when there are built-in functions to do that?

Why should I select variables due to my intuition if there are builtin functions in sklearn python like SelectKBest() and PCA() If I plot graphs of features of the data to see if they can detect the ...
1
vote
1answer
38 views

Can I apply OLS (multiple regression) to panel data to identify significant variables?

I have panel data for a 5-year period and want to explore the determinants of car prices (number of doors, house power, etc.). Is it appropriate to use OLS or multiple regression to explore the ...
0
votes
0answers
3 views

Can different data mining algorithms cross check each other's feature selection?

I have worked with the same data set for a little while, using a number of different data mining algorithms. As a result, I have developed a short list of predators which are virtually always useful - ...
0
votes
1answer
24 views

Feature selection: PCA vs intuition? [on hold]

Which one should I choose? How can I combine them (i.e. in series or parallel)? What if there are dummy features in my data? What if my intuition messes things up?
0
votes
1answer
928 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
0
votes
1answer
63 views

More features than data points in linear regression

In a dataset with more features (e.g. 120) than data points (e.g. 60) what are the techniques commonly used to select the best features to apply linear regression? Obviously there is an efficiency ...
0
votes
0answers
13 views

Cross validated loglikelihood?

This is probably a silly question: I was playing around with penalized package and cvl outputs a cross validated loglikelihood and another measure just called loglikelihood which is suppose to be ...
4
votes
1answer
779 views

Using MatchIt to match groups in a retrospective analysis

I am interested in using the R package MatchIt to preprocess my data as to obtain matched groups based on a predefined treatment variable. However I am facing a few issues. The first issue is that ...
0
votes
0answers
8 views

Results from rfe function (caret) to compute average metrics - R

I am computing a SVM-RFE model with the rfe function of the caret package, but I am a bit confused about the results. My code ...
1
vote
1answer
27 views

General-to-specific subset selection (“Autometrics”) performing well in macroeconomics

I wonder why general-to-specific (GETS) subset selection and particularly the Autometrics algorithm are performing well in macroeconomic modelling/forecasting. How does Autometrics work? Doornik ...
2
votes
1answer
178 views

Difference between selecting features based on “F regression” and based on $R^2$ values?

Is comparing features using F-regression the same as correlating features with the label individually and observing the $R^2$ value? I have often seen my ...
1
vote
0answers
33 views

Why use group lasso instead of lasso?

I have read the that the group lasso is used for variable selection and sparsity in a group of variables. I want to know the intuition behind this claim. Why is group lasso preferred to lasso? Why ...
0
votes
0answers
3 views

how to extract selectKbest feature scores from pipeline? [migrated]

My code is below. I'm wondering how to extract the selectKbest .scores_ from my pipeline, after the classifier has been trained. ...
0
votes
1answer
20 views

How to make this data in the following figure separable for the classification into three classes?

The figure below shows the PCA projections of inputs which are 14 meteorological features, (i.e. wind, temperature, humidity, pressure, and so on.) I would like to use any technique to make it more ...
2
votes
1answer
52 views

Lasso will not remove correlated variables

The very essence of lasso is that it is supposed to select only one of two correlated variables. However, when I include two highly correlated predictors (they are correlated with each other at level ...
0
votes
2answers
29 views

Is it possible to make the non-separable data more separable by any methods of feature selection, extraction or transformation?

Could these data (in the figure below) be separated by any means of feature extraction, transformation, or it's just a waste of time to make the three classes separable if they "in fact" weren't ...
0
votes
0answers
5 views

Corelations attributes with decision - multi class

I have three integer decision classes: 1,2,3. Tell me please, if it make sense to compute corelations attributes with decision, and given it select ...
0
votes
0answers
20 views

Fisher Discriminant Analysis vs ANOVA [on hold]

Both FDA and ANOVA talks about minimizing within variance and maximizing across across variance. Lets say there are 3 classes for which feature f1 data is available. We can apply both of the above ...
2
votes
1answer
521 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
6
votes
1answer
876 views

Explain steps of LLE (local linear embedding) algorithm?

I understand the basic principle behind the algorithm for LLE consists of three steps. Finding the neighborhood of each data point by some metric such as k-nn. Find weights for each neighbor which ...
0
votes
1answer
23 views

How to get the best out of a “bad” set of features for regression?

I'm trying to learn a regression model for a computer vision / pattern recognition task, where I try to estimate a continuous variable from a set of visual features. I have done preliminary ...
1
vote
0answers
21 views

Feature ranking for *known* clusters

I am aware of feature ranking (i.e. selecting the 'best' features) for a binary classification task based on some model, however, I was wondering how to do this in the absence of a model? For example, ...
0
votes
1answer
29 views

Logistic regression categorical variable interpretation after transformed into dummy variable

Before training a glm model (in R), predictors were transformed into matrix and highly correlated/near zero variance variables were excluded: ...
0
votes
1answer
28 views

Can cluster analysis of preclassified items gives idea about the classification performance?

Suppose in a classification we have a dataset with many features and their class, we want to select some features using which we can construct a classifier. We perform the cluster evaluation for the ...
2
votes
2answers
330 views

Random Forests for predictor importance (Matlab)

I'm working with a dataset of approx 150,000 observations and 50 features, using SVM for the final model. To trim down the feature count, I decided to look into using RF so SVM optimization doesn't ...
3
votes
2answers
2k views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
1
vote
0answers
29 views

Double lasso variable selection

Currently I am learning about variable selection and lasso. I found the paper by Urminsky et al. "Using Double-Lasso Regression for Principled Variable Selection" (2016) which proposes a double lasso ...
8
votes
1answer
2k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
1
vote
1answer
11 views

Method for variable shortlisting (where sample size is small and variable number is large)

I have 10 different herb samples, and each sample has ~400 chemical components in varying amounts (all numerical variables). I would like to determine which of these variables contribute to the ...
4
votes
1answer
3k views

R knn variable selection

I have a data set that's 200k rows X 50 columns. I'm trying to use a knn model on it but there is huge variance in performance depending on which variables are ...
0
votes
1answer
22 views

General Criteria for Feature Quality

It is generally accepted that the most important factor for successful machine learning is quality feature engineering: Feature Engineering is the Key At the end of the day, some machine ...
0
votes
0answers
9 views

How to utilize the features available in the train data but not in the test data to train the algorithm?

I have recently worked on some kaggle competitions and found some of them provided some features only for the training data but not for the test. For example this Expedia one does not have session ...
0
votes
1answer
31 views

Use Lasso Logistic Regression to Analyze Binary Data with

I am involved with a medical research that analyzes Coronary Artery Disease. The dataset has a couple of predictors such as age, gender, race, certain symptons and medical standard procedures to be ...
0
votes
1answer
182 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
2
votes
2answers
574 views

Event Prediction through Machine Learning

I have a large data set consisting of ca. 40 categorical data items and a few interval data items (real numbers, less than 5 such items). Most categories should have a lot of values that repeat ...
2
votes
1answer
77 views

What can be the reason to do feature selection based on variance before doing PCA?

I have noticed that when applying PCA to large datasets, people often will first subset the data considerably. Sometimes people just randomly take a subset of the features/variables, but often they ...
0
votes
0answers
8 views

Should one drop features not returned by xgb.importance?

The xgb.importance() function within R package xgboost was used. Have some questions in mind, and cannot seem to find a direct answer somewhere else, if someone can address these: When ...
3
votes
6answers
144 views

How to compare features and classifiers which achieve perfect accuracy?

So I'm looking to compare different combinations of features and classifiers. But I'm getting a lot of combinations that achieve 100% cross validation accuracy. I'm trying to figure out how I would ...
0
votes
2answers
29 views

How to combine a set of features and report their effectiveness with only one number?

I have a binary classification problem. My data set has 100 features with 10 different categories (10 features per category). I want to report the effectiveness (in terms of classification) with a ...
0
votes
0answers
17 views

Feature selection: wrapper or embedded?

I have searched for information on what could be the reasons for preferring wrapper feature selection to embedded feature selection. I found a comparative study which tells when filters are better, ...
4
votes
1answer
92 views

Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization. My question: do people normally use ...
3
votes
1answer
656 views

Linear regression for feature selection

Imagine we regress y on x1...x4. Now, we want to find out if ...
1
vote
1answer
63 views

what is “principled feature selection”?

i see the expression "principled feature selection" in titles of various Machine Learning papers and generally in the literature but nowhere do authors really define what they mean. "principled" as ...
2
votes
1answer
49 views

RReliefF algorithm for regression for feature selection with an example

How does the RReliefF algorithm for regression work? The original ReliefF algorithm for classification problems uses the concept of nearest hits and misses. I am confused how ReliefF can be used for ...
0
votes
1answer
178 views

MATLAB function TreeBagger() (Random Forest classification) and different number of variables

I am using MATLAB function TreeBagger() for Random Forest classification, for an assignment. It gives error when the number of variables of the Test data is different from the number of variables of ...
3
votes
3answers
1k views

Should feature selection be performed only on training data (or all data)?

Should be feature selection performed only on training data (or all data)? I went through some discussions and papers such as Guyon (2003) and Singhi and Liu (2006), but still not sure about right ...
0
votes
1answer
176 views

combining multiple classifiers common features

Can multiple binary-classifiers be combined to produce a final output if their feature sets have some common elements? How will this influence the accuracy?
1
vote
1answer
165 views

Python Text Classification Features Engineering

I am trying to train a model on text classification. I have a large labeled dataset. Documents are set of comments, notes on a incident. Labels are high level categories for the incidents. As ...
0
votes
1answer
17 views

How to check feature relevance and representativeness?

How to check whether features from one domain are relevant in other domain? How to evaluate whether features are representing that domain?
2
votes
2answers
1k views

WEKA: Visualize combined trees of random forest classifier

I have a small data set consisting of 385 entries and around 200 attributes. Because I want to apply attribute selection and because of the limited size of my data set, I got the advice to use the ...