Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
41 views

Feature selection and model fitness in panel data

I am interested in panel data analysis with more than 20 variables in R using the package "plm". Right now, I am looking at adjusted R-square for the set of variables that best explain my dependent ...
0
votes
1answer
28 views

Feature selection step before decision tree?

I want to use rpart (a R package) to build a decision tree model. The data is a high-dimensional expression matrix, with ~50,000 predictors and ~500 samples. The response is a categorical variable. ...
0
votes
0answers
24 views

Efficient feature selection in regression analysis

It's a Deja Vu problem but I want to discuss in a computationally efficient perspective. Assuming I am running a ordinary linear regression, I have hundreds of factors features to choose from. I want ...
0
votes
0answers
19 views

Classification on sequential data

Context: I am working on a classification project where I recommend items to customers based on their past purchase history. Question: How will "time leakage" affect training? Example: Let's say ...
1
vote
1answer
35 views

Machine learning step order question

I have been working on this project for over a year now and I believe i finally have things figured out. Mainly i'm looking for any suggestions or things i'm doing wrong with my process, but i also ...
0
votes
1answer
19 views

Picking more than desired number of features in PCA

I have encountered the presentation and one of the ideas mentioned there is as follows. Suppose, that there is a sample of objects with 100 features, only 5 of which are informative. On the 5th slide ...
1
vote
0answers
19 views

Dummies with different significance

A friend asked me this question to which I cannot answer: he is running a linear regression and he has 3 categorical independent variables which, if used altogether, would give multicollinearity. If ...
1
vote
0answers
71 views

R - suggested precedures in caret to fit stable precise binary classifiers to financial data

Building a binary precise classifier to forecast financial outcomes (stock rise vs. fall) brings up some nifty complications within caret. 1. classifier selection: there are tons of classifiers ...
0
votes
0answers
19 views

Is boosting resistant to overfitting for both number of iterations and number of features?

Boosting methods (such as the popular xgboost) do not tend to overfit when we use many iterations - Schapire and Freund. Are they also resistant to overfitting when ...
0
votes
0answers
18 views

Using PCA to determine which features are useful in classification [duplicate]

Is it possible to use PCA to determine what features can be used in classification (to determine a class)? I have a dataset consisting of 40000 observation from which 324 features are extracted. I ...
1
vote
0answers
8 views

Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions

Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?: The data points are real valued vectors in ...
0
votes
0answers
8 views

How to generate count-based features from categorical data for binary classification?

I recently discovered this blog post by Microsoft Azure. In it they describe a method of generating new count-based features from categorical features for a binary classification task. I am a bit ...
1
vote
1answer
17 views

How to map data to another feature space

I have some data which is described in a feature space $F$. Let's call this dataset $X_F$. That is, $X_F$ is a matrix where each row an instance and each column is a feature (characteristic). Suppose ...
0
votes
0answers
40 views

Understanding the approach behind variable importance returned with Xgboost method in R package caret

I recently implemented the R package caret, for a binary categorical outcome regarding a transcriptomic microarray dataset. As i used the method from the xgboost package(method="xgbtree"), then i used ...
0
votes
0answers
16 views

Rescaling vs Standardization of features

Is there any general rule of thumb or any justified rule to choose whether to scale a dataset using Rescaling (for each feature, subtract the min value and divid by the max - min) or Standardization ...
0
votes
0answers
17 views

How to select variables from data with continuous outcome/binary outcome

I'm working with a dataset containing 1000 observations and 5000 variables. And I want to select the most important variables for two outcomes: One is continuous, the other one is binary. What ...
0
votes
0answers
15 views

Any metrics on measuring importance of combinations of inputs in neural networks?

I'm not a mathematician, so I have a feeling this has an answer, but I'm probably using the wrong words. In a neural network, you have a set of input vectors ($x_{1}$, $x_{2}$, ... $x_{n}$ for $n$ ...
1
vote
0answers
77 views

Linear SVM feature weights interpretation. Binary classification, only positive feature values

I'm using clf = svm.SVC(kernel='linear') on a data set with only two classes $y \in \{-1, +1\}$ and the feature values of all samples are normalized between 0 and ...
1
vote
0answers
14 views

consistency function in FSelector

I am new in this field and I read some articles on Feature Selection. What does "consistency" function do in R's FSelector package? For instance, ...
0
votes
0answers
7 views

Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
0
votes
0answers
19 views

How can I combine binary features into higher-order combinations for Logistic Regression

I have training data which I have completely binarised, the result is 600 columns of binary features. Now I want to explore the combination of features into a single feature? Would I complete this by ...
1
vote
0answers
23 views

Preprocessing Random Forest With Lots of Features

I'm working on a project for uni where I have to predict a two-class problem, related to acceptance (or not) of a patent demand. Initially, I have a dataset separated into training and test data. My ...
0
votes
0answers
46 views

Training a CNN specifically for feature extraction

I am working on a multiclass multilabel image classification problem. I have been using pre-trained CNNs (from Caffe Model Zoo) to perform feature extraction. I then model the extracted feature ...
0
votes
0answers
17 views

Feature selection with ReliefF algorithm

I have a dataset consisting of around 10000 data points and 20 features. I'm using nested cross-validation for estimating the performance. Now, I want to do feature selection. Due to the nested ...
1
vote
0answers
30 views

Defining Importance of variables in regression and variable selection

When doing variable selection, one of the most asked questions is which variables are most important, or rank the variables in order of importance. Typically in linear or logistic regression, the ...
0
votes
0answers
17 views

Feature importance RF

What is the difference between 'DeltaCriterionDecisionSplit' in the Treebagger function and predictorImportance() function from tree ensemble in matlab? Thanks.
0
votes
1answer
31 views

Need for removing correlated and near-zero variance features despite feature selection?

I'm doing classification with two classes. Before I apply a classifier, I'm doing some preprocessing steps like removing near-zero variance features or highly correlated features (for those ...
0
votes
0answers
14 views

Influence of correlated features on classifier performance

Let's consider following example. The feature vector has N dimensions. We know that the i feature is linearly correlated to feature j. What we should do in that case. Can we neglect the j-th ...
0
votes
0answers
16 views

Create features from a document

I have been given an assignment related to NLP and I am a newbie in this field. Train a named entity recognition system that treats the documents as strings of mentions, x . A labelling of the ...
1
vote
1answer
56 views

Difference in Feature selection methods between classification and regression problems?

For high-dimensional molecular genetic data, is there a difference in available feature selection techniques between classification problems and regression problems? Or can all feature selection ...
0
votes
1answer
34 views

Model-based clustering evaluation with BIC

Let's say I have fitted two models using EM-clustering and they differ in both the number of clusters and are fitted on different subset of features (chosen from the same set of features). Could I ...
2
votes
0answers
50 views

Optimal feature selection

I am working on classification issue. My training set contains of 10D features vectors. As a training model I am going to use Fisher or Neural Network. Here is a plot of the correlation matrix for a ...
0
votes
0answers
27 views

Different variable selection techniques for Longitudinal data in R

I'm trying to perform variable selection in R and was wondering if the stepwise and Adaptive lasso codes would change for longitudinal data. Also it would be great if someone could share some sample ...
0
votes
0answers
16 views

What are the methods to measure feature relevance

I have implemented K-NN(K-nearest neighbor) algorithm and wanted to apply feature selection/weighting to it. I know some methods to measure the feature relevance such as computing the correlation ...
1
vote
0answers
13 views

Comparing categorical variable importance across groups; zero and one beta regression

I am attempting to compare behavioral responses across two species (one native and one invasive). Predictors run the range of types including continuous (size), discrete (day of trial) and ...
3
votes
1answer
58 views

Importance of variables

In a set of data, I have one dependent variable and 50 independent variables. Out of these 50, how can I find the variables which are important in estimating the dependent variable?
2
votes
1answer
188 views

Difference between selecting features based on “F regression” and based on $R^2$ values?

Is comparing features using F-regression the same as correlating features with the label individually and observing the $R^2$ value? I have often seen my ...
0
votes
0answers
22 views

Any feature evaluation method without classifier?

My question is below: In a view of pattern recognition (or machine learning), is there any method to evaluate feature vector without using classifier? For now, if i want to evaluate something new ...
1
vote
1answer
51 views

Collinearity in Classification Model for Churn Prediction

I'm working on evaluating various classification algorithms to help predict customer churn (or at least ID interesting features to use in later strategy). The goal is to identify accounts who are at ...
0
votes
1answer
30 views

Feature selection in regression with ARMA errors

I am working on creating a forecast of an auto.arima model with predictors. I consider an exploratory first step of creating simple regression models, like the ...
2
votes
0answers
21 views

Combinatorial optimization: split features into several subsets to maximize overall score

I tried to reformulate my original problem (that is quite difficult to explain) to a simpler one. Please, take it into account when you may think that the final goal doesn't make too much sense. ...
0
votes
0answers
10 views

Feature selection method dependent on response variable?

I was wondering in prediction cases where the data contains many more features than samples (the p>>n case) is the feature selection method of choice dependent on the type of response variable? For ...
0
votes
0answers
18 views

Filtering before multivariate analysis

I am a newbie in statistics and this may be a silly question. For a n << p problem, I wonder if it is feasible to first filter variables according to some criterion, for example correlation ...
0
votes
1answer
28 views

Doubt about feature selection

I'm working on a text classification problem using Python and NLTK. I've got two frequency distributions, one for each class (it's basically a binary classification). So, my doubt it's if there's a ...
3
votes
1answer
61 views

Decision Tree: Adding “important” feature doesn't necessarily improve prediction

I am using a decision tree to perform binary classification. I've found that a particular feature seems to be an important one; however, keeping it in my model doesn't yield better predictions (i.e. ...
1
vote
1answer
84 views

How do I know if I have enough features for a ML

Is there any way I can figure out if what the data I have can provide any reasonable prediction? Say, if I have 20 features, for example, how do I check that these features are actually useful for ...
0
votes
0answers
30 views

We should normalize (or standardize) data before feature selection tests (t-test, related matrix, etc.)?

We should normalize (or standardize) data before these feature selection techniques? Which one for every technique? normalization of standardization? t-test Related matrix Stepwise PCA Factor ...
2
votes
1answer
21 views

Is it necessary to use warm_start when tracking oob_score in scikit RandomForestClassifier?

I'm planning on doing feature-selection with RandomForestClassifier by using the feature_importances and ...
1
vote
1answer
32 views

How to select relevant features for binary classification from a given set of features using p values?

Lets say I have 50 features and want to select few relevant features (say around 20 features) for binary (2 class) classification. I have studied that we can use p values to decide which features are ...
3
votes
0answers
21 views

Does this pattern indicate over-fitting in machine learning?

I am working on a diagnostics project, and trying to improve the performance of a classifier(s). We have over a million features to choose from, so feature selection is a real challenge. To look ...