Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
6 views

Fixed-effects variable selection for mixed-effects regression

Does anybody know if it is possible to apply some "feature selection" algorithm to a dataset prior to creating a mixed-effects regression model? I am trying to implement such a modelling in Matlab, ...
14
votes
2answers
14k views

How to deal with multicollinearity when performing variable selection?

I have a dataset with 9 continuous independent variables. I'm trying to select amongst these variables to fit a model to a single percentage (dependent) variable, ...
0
votes
1answer
7 views

taking average of several models and feature sets

just a quick question that i cant seem to find a definitive answer for. When im doing feature selection, i end up with a list of the top performing sets. Would it make sense to use the top 10 sets ...
0
votes
0answers
8 views

Caret classification: feature selection & unbalanced data

I have a two-class classification problem with very unbalanced data (~1:1000 Yes/No ratio). The initial model class I'd like to try is regular glm. So there are two issues need to be addressed: 1) ...
0
votes
1answer
22 views

Comparing and evaluating win probabilities in sports from different settings

Background I'm trying to predict the probability that the home teams wins a certain sports game, for each minute of the game. Taking these win probabilities together produces a nice visual of the ...
0
votes
0answers
7 views

How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...
0
votes
0answers
10 views

Correct Feature Selection Methodology?

I am running a weighted multiple linear regression where my independent variables take binary values, 0 and 1. The dependent variable y, takes numeric values (positive as well as negative). The ...
4
votes
4answers
128 views

How to compare features and classifiers which achieve perfect accuracy?

So I'm looking to compare different combinations of features and classifiers. But I'm getting a lot of combinations that achieve 100% cross validation accuracy. I'm trying to figure out how I would ...
1
vote
1answer
138 views

Machine learning tutorials / examples on data sets larger than a terabyte

I am trying to gather a list of practical ML examples / tutorials on more than a terabyte of data. I'm particularly interested in feature extraction from large data sets that involves aggregation (the ...
0
votes
0answers
15 views

T-test for feature selection

I want to reduce features (voxels) in my fMRI data. I applied t-test between two conditions and select only those features which are significant (p<0.05). After that I divided the data into ...
0
votes
1answer
33 views

Main Drawbacks of stepwise regression

People typically prefer the Lasso or other methods to stepwise regression. What are the main problems in stepwise regression which makes it unreliable specifically the problems with forward selection ...
0
votes
0answers
3 views

How does Weka chiSquaredAttributeEval generates single attribute selection list while Chi Square itself is class based?

I have implemented my own Chi-Square ranker in C# however the example i found on the internet shows that Chi-Square ranks the each attribute within its class However Weka generates attributes as a ...
2
votes
1answer
144 views

Feature selection for pattern mining

I must find frequent patterns in temporal data, using a method that was imposed to me. This tool has problems handling these data: processing is long and takes a lot of memory. So, I would like to ...
0
votes
0answers
27 views

Feature selection and model fitness in panel data

I am interested in panel data analysis with more than 20 variables in R using the package "plm". Right now, I am looking at adjusted R-square for the set of variables that best explain my dependent ...
4
votes
1answer
726 views

Using MatchIt to match groups in a retrospective analysis

I am interested in using the R package MatchIt to preprocess my data as to obtain matched groups based on a predefined treatment variable. However I am facing a few issues. The first issue is that ...
1
vote
3answers
291 views

Evaluating features and similarity measures

I am currently developing a classifier, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
3
votes
1answer
673 views

Justification for feature selection by removing predictors with near zero variance

I have a large number of variables that I'm trying to reduce, and I've stumbled on Kuhn's (2008) suggestion to eliminate variables with zero or near-zero variance: [Near-zero variance means that ...
2
votes
1answer
201 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
0
votes
1answer
24 views

Feature selection step before decision tree?

I want to use rpart (a R package) to build a decision tree model. The data is a high-dimensional expression matrix, with ~50,000 predictors and ~500 samples. The response is a categorical variable. ...
0
votes
0answers
23 views

Efficient feature selection in regression analysis

It's a Deja Vu problem but I want to discuss in a computationally efficient perspective. Assuming I am running a ordinary linear regression, I have hundreds of factors features to choose from. I want ...
0
votes
0answers
17 views

Classification on sequential data

Context: I am working on a classification project where I recommend items to customers based on their past purchase history. Question: How will "time leakage" affect training? Example: Let's say ...
1
vote
1answer
29 views

Machine learning step order question

I have been working on this project for over a year now and I believe i finally have things figured out. Mainly i'm looking for any suggestions or things i'm doing wrong with my process, but i also ...
1
vote
0answers
33 views

RReliefF algorithm for regression for feature selection with an example

How does the RReliefF algorithm for regression work? The original ReliefF algorithm for classification problems uses the concept of nearest hits and misses. I am confused how ReliefF can be used for ...
0
votes
1answer
18 views

Picking more than desired number of features in PCA

I have encountered the presentation and one of the ideas mentioned there is as follows. Suppose, that there is a sample of objects with 100 features, only 5 of which are informative. On the 5th slide ...
1
vote
0answers
19 views

Dummies with different significance

A friend asked me this question to which I cannot answer: he is running a linear regression and he has 3 categorical independent variables which, if used altogether, would give multicollinearity. If ...
1
vote
1answer
16 views

How to map data to another feature space

I have some data which is described in a feature space $F$. Let's call this dataset $X_F$. That is, $X_F$ is a matrix where each row an instance and each column is a feature (characteristic). Suppose ...
1
vote
0answers
64 views

R - suggested precedures in caret to fit stable precise binary classifiers to financial data

Building a binary precise classifier to forecast financial outcomes (stock rise vs. fall) brings up some nifty complications within caret. 1. classifier selection: there are tons of classifiers ...
0
votes
0answers
16 views

Is boosting resistant to overfitting for both number of iterations and number of features?

Boosting methods (such as the popular xgboost) do not tend to overfit when we use many iterations - Schapire and Freund. Are they also resistant to overfitting when ...
0
votes
0answers
18 views

Using PCA to determine which features are useful in classification [duplicate]

Is it possible to use PCA to determine what features can be used in classification (to determine a class)? I have a dataset consisting of 40000 observation from which 324 features are extracted. I ...
0
votes
1answer
901 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
8
votes
1answer
2k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
1
vote
1answer
501 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
2
votes
1answer
140 views

Difference between selecting features based on “F regression” and based on $R^2$ values?

Is comparing features using F-regression the same as correlating features with the label individually and observing the $R^2$ value? I have often seen my ...
1
vote
0answers
8 views

Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions

Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?: The data points are real valued vectors in ...
0
votes
1answer
110 views

Feature selection with a binary dependent variable

Given we have a binary dependent variable and 100s of features and ~50k observations, is there a generally accepted way to trim the features via some type of machine learning concept? I was trying a ...
0
votes
0answers
6 views

How to generate count-based features from categorical data for binary classification?

I recently discovered this blog post by Microsoft Azure. In it they describe a method of generating new count-based features from categorical features for a binary classification task. I am a bit ...
2
votes
2answers
550 views

Event Prediction through Machine Learning

I have a large data set consisting of ca. 40 categorical data items and a few interval data items (real numbers, less than 5 such items). Most categories should have a lot of values that repeat ...
0
votes
0answers
30 views

Understanding the approach behind variable importance returned with Xgboost method in R package caret

I recently implemented the R package caret, for a binary categorical outcome regarding a transcriptomic microarray dataset. As i used the method from the xgboost package(method="xgbtree"), then i used ...
1
vote
0answers
58 views

Linear SVM feature weights interpretation. Binary classification, only positive feature values

I'm using clf = svm.SVC(kernel='linear') on a data set with only two classes $y \in \{-1, +1\}$ and the feature values of all samples are normalized between 0 and ...
2
votes
2answers
279 views

Random Forests for predictor importance (Matlab)

I'm working with a dataset of approx 150,000 observations and 50 features, using SVM for the final model. To trim down the feature count, I decided to look into using RF so SVM optimization doesn't ...
0
votes
0answers
14 views

Rescaling vs Standardization of features

Is there any general rule of thumb or any justified rule to choose whether to scale a dataset using Rescaling (for each feature, subtract the min value and divid by the max - min) or Standardization ...
0
votes
1answer
166 views

mixing binary and real-valued features with SGD

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or ...
0
votes
0answers
13 views

How to select variables from data with continuous outcome/binary outcome

I'm working with a dataset containing 1000 observations and 5000 variables. And I want to select the most important variables for two outcomes: One is continuous, the other one is binary. What ...
0
votes
0answers
15 views

Any metrics on measuring importance of combinations of inputs in neural networks?

I'm not a mathematician, so I have a feeling this has an answer, but I'm probably using the wrong words. In a neural network, you have a set of input vectors ($x_{1}$, $x_{2}$, ... $x_{n}$ for $n$ ...
1
vote
0answers
14 views

consistency function in FSelector

I am new in this field and I read some articles on Feature Selection. What does "consistency" function do in R's FSelector package? For instance, ...
0
votes
0answers
5 views

Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
0
votes
0answers
18 views

How can I combine binary features into higher-order combinations for Logistic Regression

I have training data which I have completely binarised, the result is 600 columns of binary features. Now I want to explore the combination of features into a single feature? Would I complete this by ...
1
vote
0answers
18 views

Preprocessing Random Forest With Lots of Features

I'm working on a project for uni where I have to predict a two-class problem, related to acceptance (or not) of a patent demand. Initially, I have a dataset separated into training and test data. My ...
0
votes
0answers
30 views

Training a CNN specifically for feature extraction

I am working on a multiclass multilabel image classification problem. I have been using pre-trained CNNs (from Caffe Model Zoo) to perform feature extraction. I then model the extracted feature ...
0
votes
0answers
12 views

Feature selection with ReliefF algorithm

I have a dataset consisting of around 10000 data points and 20 features. I'm using nested cross-validation for estimating the performance. Now, I want to do feature selection. Due to the nested ...