# Tagged Questions

Methods and principles of selecting a subset of attributes for use in further modelling

0answers
31 views

### Locality Sentive Hashing for Dimentionality Reduction or Feature clustering

So I have read up on LSH and Asymmetric hashing as proposed by Google for their google correlate algorithm. These work by only comparing similar items due to the multiple random hashes, however we are ...
0answers
14 views

### Fixed-effects variable selection for mixed-effects regression

Does anybody know if it is possible to apply some "feature selection" algorithm to a dataset prior to creating a mixed-effects regression model? I am trying to implement such a modelling in Matlab, ...
1answer
18 views

### taking average of several models and feature sets

just a quick question that i cant seem to find a definitive answer for. When im doing feature selection, i end up with a list of the top performing sets. Would it make sense to use the top 10 sets ...
0answers
65 views

### Caret classification: feature selection & unbalanced data

I have a two-class classification problem with very unbalanced data (~1:1000 Yes/No ratio). The initial model class I'd like to try is regular glm. So there are two issues need to be addressed: 1) ...
0answers
14 views

### How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...
0answers
16 views

### Correct Feature Selection Methodology?

I am running a weighted multiple linear regression where my independent variables take binary values, 0 and 1. The dependent variable y, takes numeric values (positive as well as negative). The ...
1answer
31 views

### Comparing and evaluating win probabilities in sports from different settings

Background I'm trying to predict the probability that the home teams wins a certain sports game, for each minute of the game. Taking these win probabilities together produces a nice visual of the ...
0answers
28 views

### T-test for feature selection

I want to reduce features (voxels) in my fMRI data. I applied t-test between two conditions and select only those features which are significant (p<0.05). After that I divided the data into ...
1answer
53 views

### Main Drawbacks of stepwise regression

People typically prefer the Lasso or other methods to stepwise regression. What are the main problems in stepwise regression which makes it unreliable specifically the problems with forward selection ...
0answers
26 views

### How does Weka chiSquaredAttributeEval generates single attribute selection list while Chi Square itself is class based?

I have implemented my own Chi-Square ranker in C# however the example i found on the internet shows that Chi-Square ranks the each attribute within its class However Weka generates attributes as a ...
0answers
62 views

### Feature selection and model fitness in panel data

I am interested in panel data analysis with more than 20 variables in R using the package "plm". Right now, I am looking at adjusted R-square for the set of variables that best explain my dependent ...
1answer
37 views

### Feature selection step before decision tree?

I want to use rpart (a R package) to build a decision tree model. The data is a high-dimensional expression matrix, with ~50,000 predictors and ~500 samples. The response is a categorical variable. ...
0answers
25 views

### Efficient feature selection in regression analysis

It's a Deja Vu problem but I want to discuss in a computationally efficient perspective. Assuming I am running a ordinary linear regression, I have hundreds of factors features to choose from. I want ...
0answers
27 views

### Classification on sequential data

Context: I am working on a classification project where I recommend items to customers based on their past purchase history. Question: How will "time leakage" affect training? Example: Let's say ...
1answer
42 views

### Machine learning step order question

I have been working on this project for over a year now and I believe i finally have things figured out. Mainly i'm looking for any suggestions or things i'm doing wrong with my process, but i also ...
1answer
20 views

### Picking more than desired number of features in PCA

I have encountered the presentation and one of the ideas mentioned there is as follows. Suppose, that there is a sample of objects with 100 features, only 5 of which are informative. On the 5th slide ...
0answers
20 views

### Dummies with different significance

A friend asked me this question to which I cannot answer: he is running a linear regression and he has 3 categorical independent variables which, if used altogether, would give multicollinearity. If ...
0answers
86 views

### R - suggested precedures in caret to fit stable precise binary classifiers to financial data

Building a binary precise classifier to forecast financial outcomes (stock rise vs. fall) brings up some nifty complications within caret. 1. classifier selection: there are tons of classifiers ...
0answers
24 views

### Is boosting resistant to overfitting for both number of iterations and number of features?

Boosting methods (such as the popular xgboost) do not tend to overfit when we use many iterations - Schapire and Freund. Are they also resistant to overfitting when ...
0answers
18 views

### Using PCA to determine which features are useful in classification [duplicate]

Is it possible to use PCA to determine what features can be used in classification (to determine a class)? I have a dataset consisting of 40000 observation from which 324 features are extracted. I ...
0answers
10 views

### Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions

Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?: The data points are real valued vectors in ...
0answers
9 views

### How to generate count-based features from categorical data for binary classification?

I recently discovered this blog post by Microsoft Azure. In it they describe a method of generating new count-based features from categorical features for a binary classification task. I am a bit ...
1answer
19 views

### How to map data to another feature space

I have some data which is described in a feature space $F$. Let's call this dataset $X_F$. That is, $X_F$ is a matrix where each row an instance and each column is a feature (characteristic). Suppose ...
0answers
70 views

### Understanding the approach behind variable importance returned with Xgboost method in R package caret

I recently implemented the R package caret, for a binary categorical outcome regarding a transcriptomic microarray dataset. As i used the method from the xgboost package(method="xgbtree"), then i used ...
0answers
18 views

### Rescaling vs Standardization of features

Is there any general rule of thumb or any justified rule to choose whether to scale a dataset using Rescaling (for each feature, subtract the min value and divid by the max - min) or Standardization (...
0answers
20 views

### How to select variables from data with continuous outcome/binary outcome

I'm working with a dataset containing 1000 observations and 5000 variables. And I want to select the most important variables for two outcomes: One is continuous, the other one is binary. What ...
0answers
17 views

### Any metrics on measuring importance of combinations of inputs in neural networks?

I'm not a mathematician, so I have a feeling this has an answer, but I'm probably using the wrong words. In a neural network, you have a set of input vectors ($x_{1}$, $x_{2}$, ... $x_{n}$ for $n$ ...
0answers
118 views

### Linear SVM feature weights interpretation. Binary classification, only positive feature values

I'm using clf = svm.SVC(kernel='linear') on a data set with only two classes $y \in \{-1, +1\}$ and the feature values of all samples are normalized between 0 and 1....
0answers
22 views

### consistency function in FSelector

I am new in this field and I read some articles on Feature Selection. What does "consistency" function do in R's FSelector package? For instance, ...
0answers
13 views

### Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
0answers
42 views

### How can I combine binary features into higher-order combinations for Logistic Regression

I have training data which I have completely binarised, the result is 600 columns of binary features. Now I want to explore the combination of features into a single feature? Would I complete this by ...
0answers
34 views

### Preprocessing Random Forest With Lots of Features

I'm working on a project for uni where I have to predict a two-class problem, related to acceptance (or not) of a patent demand. Initially, I have a dataset separated into training and test data. My ...
0answers
108 views

### Training a CNN specifically for feature extraction

I am working on a multiclass multilabel image classification problem. I have been using pre-trained CNNs (from Caffe Model Zoo) to perform feature extraction. I then model the extracted feature ...
0answers
27 views

### Feature selection with ReliefF algorithm

I have a dataset consisting of around 10000 data points and 20 features. I'm using nested cross-validation for estimating the performance. Now, I want to do feature selection. Due to the nested cross-...
0answers
39 views

### Defining Importance of variables in regression and variable selection

When doing variable selection, one of the most asked questions is which variables are most important, or rank the variables in order of importance. Typically in linear or logistic regression, the ...
0answers
21 views

### Feature importance RF

What is the difference between 'DeltaCriterionDecisionSplit' in the Treebagger function and predictorImportance() function from tree ensemble in matlab? Thanks.
1answer
39 views

### Need for removing correlated and near-zero variance features despite feature selection?

I'm doing classification with two classes. Before I apply a classifier, I'm doing some preprocessing steps like removing near-zero variance features or highly correlated features (for those ...
0answers
15 views

### Influence of correlated features on classifier performance

Let's consider following example. The feature vector has N dimensions. We know that the i feature is linearly correlated to feature j. What we should do in that case. Can we neglect the j-th ...
0answers
18 views

### Create features from a document

I have been given an assignment related to NLP and I am a newbie in this field. Train a named entity recognition system that treats the documents as strings of mentions, x . A labelling of the ...
1answer
61 views

### Difference in Feature selection methods between classification and regression problems?

For high-dimensional molecular genetic data, is there a difference in available feature selection techniques between classification problems and regression problems? Or can all feature selection ...
1answer
38 views

### Model-based clustering evaluation with BIC

Let's say I have fitted two models using EM-clustering and they differ in both the number of clusters and are fitted on different subset of features (chosen from the same set of features). Could I ...
0answers
64 views

### Optimal feature selection

I am working on classification issue. My training set contains of 10D features vectors. As a training model I am going to use Fisher or Neural Network. Here is a plot of the correlation matrix for a ...
0answers
48 views

### Different variable selection techniques for Longitudinal data in R

I'm trying to perform variable selection in R and was wondering if the stepwise and Adaptive lasso codes would change for longitudinal data. Also it would be great if someone could share some sample ...
0answers
26 views

### What are the methods to measure feature relevance

I have implemented K-NN(K-nearest neighbor) algorithm and wanted to apply feature selection/weighting to it. I know some methods to measure the feature relevance such as computing the correlation ...
0answers
13 views

### Comparing categorical variable importance across groups; zero and one beta regression

I am attempting to compare behavioral responses across two species (one native and one invasive). Predictors run the range of types including continuous (size), discrete (day of trial) and ...
1answer
60 views

### Importance of variables

In a set of data, I have one dependent variable and 50 independent variables. Out of these 50, how can I find the variables which are important in estimating the dependent variable?
1answer
352 views

### Difference between selecting features based on “F regression” and based on $R^2$ values?

Is comparing features using F-regression the same as correlating features with the label individually and observing the $R^2$ value? I have often seen my ...
0answers
25 views

### Any feature evaluation method without classifier?

My question is below: In a view of pattern recognition (or machine learning), is there any method to evaluate feature vector without using classifier? For now, if i want to evaluate something new ...
1answer
82 views

### Collinearity in Classification Model for Churn Prediction

I'm working on evaluating various classification algorithms to help predict customer churn (or at least ID interesting features to use in later strategy). The goal is to identify accounts who are at ...
1answer
40 views

### Feature selection in regression with ARMA errors

I am working on creating a forecast of an auto.arima model with predictors. I consider an exploratory first step of creating simple regression models, like the ...