Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
0answers
21 views

Feature ranking for *known* clusters

I am aware of feature ranking (i.e. selecting the 'best' features) for a binary classification task based on some model, however, I was wondering how to do this in the absence of a model? For example, ...
0
votes
1answer
32 views

How to get the best out of a “bad” set of features for regression?

I'm trying to learn a regression model for a computer vision / pattern recognition task, where I try to estimate a continuous variable from a set of visual features. I have done preliminary ...
1
vote
0answers
40 views

Double lasso variable selection

Currently I am learning about variable selection and lasso. I found the paper by Urminsky et al. "Using Double-Lasso Regression for Principled Variable Selection" (2016) which proposes a double lasso ...
0
votes
1answer
35 views

Can cluster analysis of preclassified items gives idea about the classification performance?

Suppose in a classification we have a dataset with many features and their class, we want to select some features using which we can construct a classifier. We perform the cluster evaluation for the ...
0
votes
1answer
32 views

Logistic regression categorical variable interpretation after transformed into dummy variable

Before training a glm model (in R), predictors were transformed into matrix and highly correlated/near zero variance variables were excluded: ...
1
vote
1answer
13 views

Method for variable shortlisting (where sample size is small and variable number is large)

I have 10 different herb samples, and each sample has ~400 chemical components in varying amounts (all numerical variables). I would like to determine which of these variables contribute to the ...
0
votes
1answer
24 views

General Criteria for Feature Quality

It is generally accepted that the most important factor for successful machine learning is quality feature engineering: Feature Engineering is the Key At the end of the day, some machine ...
0
votes
0answers
10 views

How to utilize the features available in the train data but not in the test data to train the algorithm?

I have recently worked on some kaggle competitions and found some of them provided some features only for the training data but not for the test. For example this Expedia one does not have session ...
0
votes
1answer
39 views

Use Lasso Logistic Regression to Analyze Binary Data with

I am involved with a medical research that analyzes Coronary Artery Disease. The dataset has a couple of predictors such as age, gender, race, certain symptons and medical standard procedures to be ...
1
vote
1answer
52 views

Can I apply OLS (multiple regression) to panel data to identify significant variables?

I have panel data for a 5-year period and want to explore the determinants of car prices (number of doors, house power, etc.). Is it appropriate to use OLS or multiple regression to explore the ...
0
votes
0answers
15 views

Should one drop features not returned by xgb.importance?

The xgb.importance() function within R package xgboost was used. Have some questions in mind, and cannot seem to find a direct answer somewhere else, if someone can address these: When xgb....
0
votes
0answers
26 views

Feature selection: wrapper or embedded?

I have searched for information on what could be the reasons for preferring wrapper feature selection to embedded feature selection. I found a comparative study which tells when filters are better, ...
2
votes
1answer
84 views

What can be the reason to do feature selection based on variance before doing PCA?

I have noticed that when applying PCA to large datasets, people often will first subset the data considerably. Sometimes people just randomly take a subset of the features/variables, but often they ...
4
votes
1answer
112 views

Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization. My question: do people normally use ...
1
vote
1answer
67 views

what is “principled feature selection”?

i see the expression "principled feature selection" in titles of various Machine Learning papers and generally in the literature but nowhere do authors really define what they mean. "principled" as ...
0
votes
2answers
31 views

How to combine a set of features and report their effectiveness with only one number?

I have a binary classification problem. My data set has 100 features with 10 different categories (10 features per category). I want to report the effectiveness (in terms of classification) with a ...
0
votes
0answers
33 views

Random Forest: Strange Feature Importance Results for Constant Variables

I've been using the RandomForestClassifier in python's Sklearn package to assess the importance of the features in a large dataset with features that are both binary and continuous. I've done quite a ...
0
votes
0answers
26 views

Interpreting results from lasso regression?

I have a time series data set with about 2million observations and 31 variables, which I break to a few thousand using threshold value for my dependent variable. I am using lasso regression in R to ...
1
vote
1answer
39 views

How to determine which variables to be used for cluster analysis

I have about 10 variables (features) and want to do cluster analysis of cases (data points). I have a number of ideas about which variables to be included for cluster analysis: Plot the variables ...
0
votes
0answers
27 views

Locality Sentive Hashing for Dimentionality Reduction or Feature clustering

So I have read up on LSH and Asymmetric hashing as proposed by Google for their google correlate algorithm. These work by only comparing similar items due to the multiple random hashes, however we are ...
0
votes
0answers
11 views

Fixed-effects variable selection for mixed-effects regression

Does anybody know if it is possible to apply some "feature selection" algorithm to a dataset prior to creating a mixed-effects regression model? I am trying to implement such a modelling in Matlab, ...
0
votes
1answer
15 views

taking average of several models and feature sets

just a quick question that i cant seem to find a definitive answer for. When im doing feature selection, i end up with a list of the top performing sets. Would it make sense to use the top 10 sets ...
0
votes
0answers
54 views

Caret classification: feature selection & unbalanced data

I have a two-class classification problem with very unbalanced data (~1:1000 Yes/No ratio). The initial model class I'd like to try is regular glm. So there are two issues need to be addressed: 1) ...
0
votes
0answers
13 views

How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...
0
votes
0answers
14 views

Correct Feature Selection Methodology?

I am running a weighted multiple linear regression where my independent variables take binary values, 0 and 1. The dependent variable y, takes numeric values (positive as well as negative). The ...
0
votes
1answer
29 views

Comparing and evaluating win probabilities in sports from different settings

Background I'm trying to predict the probability that the home teams wins a certain sports game, for each minute of the game. Taking these win probabilities together produces a nice visual of the ...
0
votes
0answers
27 views

T-test for feature selection

I want to reduce features (voxels) in my fMRI data. I applied t-test between two conditions and select only those features which are significant (p<0.05). After that I divided the data into ...
0
votes
1answer
50 views

Main Drawbacks of stepwise regression

People typically prefer the Lasso or other methods to stepwise regression. What are the main problems in stepwise regression which makes it unreliable specifically the problems with forward selection ...
0
votes
0answers
18 views

How does Weka chiSquaredAttributeEval generates single attribute selection list while Chi Square itself is class based?

I have implemented my own Chi-Square ranker in C# however the example i found on the internet shows that Chi-Square ranks the each attribute within its class However Weka generates attributes as a ...
0
votes
0answers
48 views

Feature selection and model fitness in panel data

I am interested in panel data analysis with more than 20 variables in R using the package "plm". Right now, I am looking at adjusted R-square for the set of variables that best explain my dependent ...
0
votes
1answer
34 views

Feature selection step before decision tree?

I want to use rpart (a R package) to build a decision tree model. The data is a high-dimensional expression matrix, with ~50,000 predictors and ~500 samples. The response is a categorical variable. ...
0
votes
0answers
25 views

Efficient feature selection in regression analysis

It's a Deja Vu problem but I want to discuss in a computationally efficient perspective. Assuming I am running a ordinary linear regression, I have hundreds of factors features to choose from. I want ...
0
votes
0answers
22 views

Classification on sequential data

Context: I am working on a classification project where I recommend items to customers based on their past purchase history. Question: How will "time leakage" affect training? Example: Let's say ...
1
vote
1answer
40 views

Machine learning step order question

I have been working on this project for over a year now and I believe i finally have things figured out. Mainly i'm looking for any suggestions or things i'm doing wrong with my process, but i also ...
0
votes
1answer
20 views

Picking more than desired number of features in PCA

I have encountered the presentation and one of the ideas mentioned there is as follows. Suppose, that there is a sample of objects with 100 features, only 5 of which are informative. On the 5th slide ...
1
vote
0answers
20 views

Dummies with different significance

A friend asked me this question to which I cannot answer: he is running a linear regression and he has 3 categorical independent variables which, if used altogether, would give multicollinearity. If ...
1
vote
0answers
80 views

R - suggested precedures in caret to fit stable precise binary classifiers to financial data

Building a binary precise classifier to forecast financial outcomes (stock rise vs. fall) brings up some nifty complications within caret. 1. classifier selection: there are tons of classifiers ...
0
votes
0answers
22 views

Is boosting resistant to overfitting for both number of iterations and number of features?

Boosting methods (such as the popular xgboost) do not tend to overfit when we use many iterations - Schapire and Freund. Are they also resistant to overfitting when ...
0
votes
0answers
18 views

Using PCA to determine which features are useful in classification [duplicate]

Is it possible to use PCA to determine what features can be used in classification (to determine a class)? I have a dataset consisting of 40000 observation from which 324 features are extracted. I ...
1
vote
0answers
9 views

Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions

Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?: The data points are real valued vectors in ...
0
votes
0answers
9 views

How to generate count-based features from categorical data for binary classification?

I recently discovered this blog post by Microsoft Azure. In it they describe a method of generating new count-based features from categorical features for a binary classification task. I am a bit ...
1
vote
1answer
18 views

How to map data to another feature space

I have some data which is described in a feature space $F$. Let's call this dataset $X_F$. That is, $X_F$ is a matrix where each row an instance and each column is a feature (characteristic). Suppose ...
0
votes
0answers
51 views

Understanding the approach behind variable importance returned with Xgboost method in R package caret

I recently implemented the R package caret, for a binary categorical outcome regarding a transcriptomic microarray dataset. As i used the method from the xgboost package(method="xgbtree"), then i used ...
0
votes
0answers
17 views

Rescaling vs Standardization of features

Is there any general rule of thumb or any justified rule to choose whether to scale a dataset using Rescaling (for each feature, subtract the min value and divid by the max - min) or Standardization (...
0
votes
0answers
18 views

How to select variables from data with continuous outcome/binary outcome

I'm working with a dataset containing 1000 observations and 5000 variables. And I want to select the most important variables for two outcomes: One is continuous, the other one is binary. What ...
1
vote
0answers
15 views

Any metrics on measuring importance of combinations of inputs in neural networks?

I'm not a mathematician, so I have a feeling this has an answer, but I'm probably using the wrong words. In a neural network, you have a set of input vectors ($x_{1}$, $x_{2}$, ... $x_{n}$ for $n$ ...
1
vote
0answers
91 views

Linear SVM feature weights interpretation. Binary classification, only positive feature values

I'm using clf = svm.SVC(kernel='linear') on a data set with only two classes $y \in \{-1, +1\}$ and the feature values of all samples are normalized between 0 and 1....
1
vote
0answers
21 views

consistency function in FSelector

I am new in this field and I read some articles on Feature Selection. What does "consistency" function do in R's FSelector package? For instance, ...
0
votes
0answers
12 views

Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
0
votes
0answers
29 views

How can I combine binary features into higher-order combinations for Logistic Regression

I have training data which I have completely binarised, the result is 600 columns of binary features. Now I want to explore the combination of features into a single feature? Would I complete this by ...