Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
0answers
15 views

Any metrics on measuring importance of combinations of inputs in neural networks?

I'm not a mathematician, so I have a feeling this has an answer, but I'm probably using the wrong words. In a neural network, you have a set of input vectors ($x_{1}$, $x_{2}$, ... $x_{n}$ for $n$ ...
1
vote
0answers
93 views

Linear SVM feature weights interpretation. Binary classification, only positive feature values

I'm using clf = svm.SVC(kernel='linear') on a data set with only two classes $y \in \{-1, +1\}$ and the feature values of all samples are normalized between 0 and 1....
1
vote
0answers
22 views

consistency function in FSelector

I am new in this field and I read some articles on Feature Selection. What does "consistency" function do in R's FSelector package? For instance, ...
0
votes
0answers
12 views

Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
0
votes
0answers
30 views

How can I combine binary features into higher-order combinations for Logistic Regression

I have training data which I have completely binarised, the result is 600 columns of binary features. Now I want to explore the combination of features into a single feature? Would I complete this by ...
1
vote
0answers
27 views

Preprocessing Random Forest With Lots of Features

I'm working on a project for uni where I have to predict a two-class problem, related to acceptance (or not) of a patent demand. Initially, I have a dataset separated into training and test data. My ...
0
votes
0answers
82 views

Training a CNN specifically for feature extraction

I am working on a multiclass multilabel image classification problem. I have been using pre-trained CNNs (from Caffe Model Zoo) to perform feature extraction. I then model the extracted feature ...
0
votes
0answers
25 views

Feature selection with ReliefF algorithm

I have a dataset consisting of around 10000 data points and 20 features. I'm using nested cross-validation for estimating the performance. Now, I want to do feature selection. Due to the nested cross-...
1
vote
0answers
36 views

Defining Importance of variables in regression and variable selection

When doing variable selection, one of the most asked questions is which variables are most important, or rank the variables in order of importance. Typically in linear or logistic regression, the ...
0
votes
0answers
18 views

Feature importance RF

What is the difference between 'DeltaCriterionDecisionSplit' in the Treebagger function and predictorImportance() function from tree ensemble in matlab? Thanks.
0
votes
1answer
33 views

Need for removing correlated and near-zero variance features despite feature selection?

I'm doing classification with two classes. Before I apply a classifier, I'm doing some preprocessing steps like removing near-zero variance features or highly correlated features (for those ...
0
votes
0answers
15 views

Influence of correlated features on classifier performance

Let's consider following example. The feature vector has N dimensions. We know that the i feature is linearly correlated to feature j. What we should do in that case. Can we neglect the j-th ...
0
votes
0answers
18 views

Create features from a document

I have been given an assignment related to NLP and I am a newbie in this field. Train a named entity recognition system that treats the documents as strings of mentions, x . A labelling of the ...
1
vote
1answer
59 views

Difference in Feature selection methods between classification and regression problems?

For high-dimensional molecular genetic data, is there a difference in available feature selection techniques between classification problems and regression problems? Or can all feature selection ...
0
votes
1answer
36 views

Model-based clustering evaluation with BIC

Let's say I have fitted two models using EM-clustering and they differ in both the number of clusters and are fitted on different subset of features (chosen from the same set of features). Could I ...
2
votes
0answers
58 views

Optimal feature selection

I am working on classification issue. My training set contains of 10D features vectors. As a training model I am going to use Fisher or Neural Network. Here is a plot of the correlation matrix for a ...
0
votes
0answers
40 views

Different variable selection techniques for Longitudinal data in R

I'm trying to perform variable selection in R and was wondering if the stepwise and Adaptive lasso codes would change for longitudinal data. Also it would be great if someone could share some sample ...
0
votes
0answers
22 views

What are the methods to measure feature relevance

I have implemented K-NN(K-nearest neighbor) algorithm and wanted to apply feature selection/weighting to it. I know some methods to measure the feature relevance such as computing the correlation ...
1
vote
0answers
13 views

Comparing categorical variable importance across groups; zero and one beta regression

I am attempting to compare behavioral responses across two species (one native and one invasive). Predictors run the range of types including continuous (size), discrete (day of trial) and ...
3
votes
1answer
59 views

Importance of variables

In a set of data, I have one dependent variable and 50 independent variables. Out of these 50, how can I find the variables which are important in estimating the dependent variable?
2
votes
1answer
268 views

Difference between selecting features based on “F regression” and based on $R^2$ values?

Is comparing features using F-regression the same as correlating features with the label individually and observing the $R^2$ value? I have often seen my ...
0
votes
0answers
22 views

Any feature evaluation method without classifier?

My question is below: In a view of pattern recognition (or machine learning), is there any method to evaluate feature vector without using classifier? For now, if i want to evaluate something new ...
1
vote
1answer
69 views

Collinearity in Classification Model for Churn Prediction

I'm working on evaluating various classification algorithms to help predict customer churn (or at least ID interesting features to use in later strategy). The goal is to identify accounts who are at ...
0
votes
1answer
38 views

Feature selection in regression with ARMA errors

I am working on creating a forecast of an auto.arima model with predictors. I consider an exploratory first step of creating simple regression models, like the ...
2
votes
0answers
24 views

Combinatorial optimization: split features into several subsets to maximize overall score

I tried to reformulate my original problem (that is quite difficult to explain) to a simpler one. Please, take it into account when you may think that the final goal doesn't make too much sense. ...
0
votes
0answers
11 views

Feature selection method dependent on response variable?

I was wondering in prediction cases where the data contains many more features than samples (the p>>n case) is the feature selection method of choice dependent on the type of response variable? For ...
0
votes
0answers
21 views

Filtering before multivariate analysis

I am a newbie in statistics and this may be a silly question. For a n << p problem, I wonder if it is feasible to first filter variables according to some criterion, for example correlation ...
0
votes
1answer
31 views

Doubt about feature selection

I'm working on a text classification problem using Python and NLTK. I've got two frequency distributions, one for each class (it's basically a binary classification). So, my doubt it's if there's a ...
3
votes
1answer
64 views

Decision Tree: Adding “important” feature doesn't necessarily improve prediction

I am using a decision tree to perform binary classification. I've found that a particular feature seems to be an important one; however, keeping it in my model doesn't yield better predictions (i.e. ...
1
vote
1answer
89 views

How do I know if I have enough features for a ML

Is there any way I can figure out if what the data I have can provide any reasonable prediction? Say, if I have 20 features, for example, how do I check that these features are actually useful for ...
0
votes
0answers
45 views

We should normalize (or standardize) data before feature selection tests (t-test, related matrix, etc.)?

We should normalize (or standardize) data before these feature selection techniques? Which one for every technique? normalization of standardization? t-test Related matrix Stepwise PCA Factor ...
2
votes
1answer
30 views

Is it necessary to use warm_start when tracking oob_score in scikit RandomForestClassifier?

I'm planning on doing feature-selection with RandomForestClassifier by using the feature_importances and ...
1
vote
1answer
41 views

How to select relevant features for binary classification from a given set of features using p values?

Lets say I have 50 features and want to select few relevant features (say around 20 features) for binary (2 class) classification. I have studied that we can use p values to decide which features are ...
3
votes
0answers
22 views

Does this pattern indicate over-fitting in machine learning?

I am working on a diagnostics project, and trying to improve the performance of a classifier(s). We have over a million features to choose from, so feature selection is a real challenge. To look ...
5
votes
3answers
229 views

For linear classifiers, do larger coefficients imply more important features?

I'm a software engineer working on machine learning. From my understanding, linear regression (such as OLS) and linear classification (such as logistic regression and SVM) make a prediction based on ...
3
votes
1answer
102 views

To select variables or not in logistic regression

I am trying to find predictors for an outcome. I was taught to perform univariate analyses & put significant variables into a multivariate logistic regression model. Then I remove variables one by ...
0
votes
1answer
74 views

What is the best way to select variables for clogit model?

I am doing clogit model (clogit of survival package) with around 150 independent variables which are highly correlated. I have to select the combinations of the variables so that the model will be the ...
0
votes
0answers
27 views

Forward search feature selection and cross-validation

I've a question regarding forward search for feature selection. Basically, I've found here and here that the procedure is the following: As the procedure suggests, the cross-validation is applied ...
0
votes
1answer
47 views

building up a predictive model with lots of features and missing data

I'm learning using R to build predictive models recently by myself and have many questions on how to attack a question. I'm given a data set of 8000 observations with 300 features. My goal is to build ...
3
votes
0answers
19 views

Feature Selection Among Groups

I'm trying to do feature selection along a dataset which has: ...
0
votes
0answers
13 views

Feature selection using GA with cross validation

I am working with feature selection on a dataset with 2100 instances.I have split my dataset into training(75%) and testing set(25%).I am using genetic algorithm for finding the optimal feature subset ...
2
votes
0answers
152 views

Coalitional effect in logistic regression and assessing explanarory variable contribution

I have a problem that could be described as logistic regression with all dichotomous variables: 1 response variable (DV) Y (I would call it later as a feature/violet star) and 5 explanatory variables (...
0
votes
0answers
14 views

Polynomial features on logistic regression [duplicate]

I'm working on a LR model, I'm currently trying to add some polynomial features of degree 2. Since the next step is choosing which polynomial features I have to discard, I've got a question: if I keep ...
1
vote
1answer
40 views

Randomizing Class Labels during classification to asses the feature selection results

I have a binary classification problem with thousands of variables and less than a hundred data points and class labels. The class is imbalanced (24 positive 51 negative samples). I have selected some ...
0
votes
0answers
29 views

Kernel PCA increases dimensionality compared with PCA?

I was trying to use sklearn to perform kernel PCA with 28*28 = 784 dims data. At first I used PCA to reduce dimensionality and I chose to reduce to k dimensions where k could explain 95% of the ...
0
votes
1answer
67 views

Feature Selection for a Machine Learning problem

I have a Machine Learning problem at hand but I'm not sure how to approach it. I have a dataset which has around 5000 observations and around 250 features(most of them are numeric and around 3-4 are ...
5
votes
2answers
114 views

What's wrong with data-guided modeling in regression?

In the Regression Modelling Strategies of Frank Harrell, section 4.1, if I understood correctly, it is not recommendred to using the data to decide how to represent a predictor in a regression model (...
1
vote
1answer
106 views

Does it makes sense to use feature selection before Random Forest?

Everything is in the title, does it makes sense to use feature selection before using random forest? Thank you
0
votes
3answers
113 views

Suspicious results after clustering

I've done a clustering and I think that my results are too good to be trusted. Here is my pipeline: Inputs: a dataset of 208 images, distributed into 2 classes (99 and 109 images in each class). ...
0
votes
0answers
101 views

Lag order selection in error correction model (ECM)

I am building an Error Correction Model for monthly price data ($X, Y, Z$). I am deliberately using an ECM and not VECM and apply a two step approach (estimating cointegration relationship first, then ...