0
votes
1answer
19 views

Updating set of probabilities for sampling with features importance

I'm currently working on some algorithm and I'm kinda out of idea for a problem I'm trying to tacle. Basically I'm trying to subsample the features of a dataset. I want to subsample that given this ...
0
votes
0answers
30 views

HIstogram of oriented gradients (HOG) features descriptor theoretical problems

I'm going to implement HOG as my features descriptor. But there are some things that make me confused: For example: If we have an image with size of 10 x 20 If we want to compute the HOG of that ...
0
votes
1answer
29 views

Grid Search for hyperparameter and feature selection

So I need to select my hyperparameters and also my features. A full grid search of the space of hyperparameters and features is too computationally intensive, so what I am doing instead is for each ...
1
vote
0answers
23 views

How to determine appropriate number of features and also which features to select?

So I have a dataset which I am using K fold cross validation on to select the number of features and which features should be selected. As I understand it, I would set the number of features to be ...
2
votes
1answer
34 views

In decision tree construction, can a good splitter have low information gain?

I have a data set with a candidate splitter variable that is a natural choice from the business perspective. It has two values, and the distributions of the target when conditioned on the two values ...
3
votes
1answer
62 views

SVM basic theory?

I have some questions about SVM: In SVM there is a nonlinear and linear SVM. What is the difference between them? To do classification in SVM, we will find the linearly separable boundary ...
0
votes
0answers
19 views

Segmenting an interval sensibly

Is there a canonical/recommended approach to or algorithm for splitting up an interval with the intent of minimizing the number of segments while keeping a high accuracy? It is essentially an ...
0
votes
1answer
66 views

Good algorithms for feature extraction from images?

I am searching for some algorithms for feature extraction from images which I want to classify using machine learning . I have heard only about [scale-invariant feature transform][1] (SIFT), I have ...
0
votes
1answer
36 views

How to compare features and classifiers which achieve perfect accuracy?

So I'm looking to compare different combinations of features and classifiers. But I'm getting a lot of combinations that achieve 100% cross validation accuracy. I'm trying to figure out how I would ...
2
votes
3answers
21 views

How to compare features and classifiers which achieve perfect accuracy?

So I'm looking to compare different combinations of features and classifiers. But I'm getting a lot of combinations that achieve 100% cross validation accuracy. I'm trying to figure out how I would ...
0
votes
0answers
19 views

Proper way to determine attribute feature selection's smaller subset based on result metrics

Overview My goal is to predict survival of an instance for five different time periods (binary attribute). I have a 100,000-instance dataset with 40 attributes and I want to reduce the attributes ...
0
votes
0answers
38 views

Recursive feature elimination with only two classes

Recursive feature elimination (RFE) is a feature-selection strategy. It performs in two nested levels of cross-validation. First it tries to divide the training set into N folds. RFE puts one fold ...
3
votes
1answer
147 views

Understanding the output of C5.0 classification model using the CARET package

The C5.0 classification model was used in this 4-class problem data with $N_{train}$=165, $P$=11, using caret R-package by ...
0
votes
0answers
39 views

equivalent of PCA explained variance ratio for SVD ?

i am wondering if there is an equivalent of PCA explained variance ratio for SVD. What are the measures I can get to monitor the number of columns I keep after the SVD ? Are any of these metrics ...
3
votes
1answer
78 views

Explain steps of LLE (local linear embedding) algorithm?

I understand the basic principle behind the algorithm for LLE consists of three steps. Finding the neighborhood of each data point by some metric such as k-nn. Find weights for each neighbor which ...
2
votes
2answers
83 views

improve precision in text classification

I am working on binary text classification using sklearn: The length of each sample is not high (~ 200-500 characters) I use TF-IDF to get important words as TfidfVectorizer(sublinear_tf=False, ...
0
votes
3answers
86 views

Classification performance and the feature set selection

I am now working on a classification problem. The generated feature set can be separated into two group. I did a comparison study: use all of the features; use the features of group 1 only; and use ...
0
votes
1answer
123 views

Feature selection and cross validation

I'm working on a project and I would like to know if the following strategy is good/correct. Sorry if this is a basic/stupid idea (I'm new to this). The input is a dataset with 2.500 features and ...
0
votes
0answers
17 views

Comparing 2 different sets of features

So let's say I have two different sets of features A and B. I'm trying to determine which set of features is the best. I'm using leave-one-out cross validation as the final metric as my data set is ...
1
vote
0answers
56 views

How to explain difference of importance between feature selection and model quality?

I have a data collection with a mixed feature set consisting of both numerical features and text features. The number of numerical features is quite small, i.e., 6, comparing to the number of text ...
0
votes
0answers
34 views

Predictive features with high presence in one class

I am doing a logistic regression to predict the outcome of a binary variable, say whether a journal paper gets accepted or not. The independent variable or predictors are all the phrases used in these ...
2
votes
3answers
210 views

How to divide feature set for selection and training

I have training data with 260 observations that have a total of 7 classes. Each observation has 120 features. I applied feature selection based on the Bhattacharyya Algorithm and got the top 40 ...
0
votes
0answers
82 views

Find weight of features for feature selection

I have a data set of videos from which I need to recognize the emotion of the speaker. For that reason I have some markers on the face of the speaker. I detect their movement as the speaker speaks and ...
2
votes
2answers
133 views

Variable selection with groups of predictors that are highly correlated

What variable selection approach should I consider if I have thousands of predictors with clusters that are extremely correlated? For example I might have a predictor set $X:= ...
2
votes
1answer
47 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
1
vote
1answer
147 views

Decision trees variable (feature) scaling and variable (feature) normalization (tuning) required in which implementations?

In many machine learning algorithms, feature scaling (aka variable scaling, normalization) is a common prepocessing step Wikipedia - Feature Scaling -- this question was close Question#41704 - How and ...
0
votes
4answers
118 views

Machine learning input relationships

After learning about a few machine-learning models (NN, SVM, decision trees), I was wondering if these models are able to find inherent relationships when learning. For example, if I feed it two ...
0
votes
0answers
97 views

Time series classification

I am classifying a set of time series inputs after creating independent features from every $n$ samples and running machine learning algorithms. I get good accuracy based on many error metrics on the ...
0
votes
0answers
113 views

Weak Classifiers weights/contribution in Adaboost and Real adaboost?

In Adaboost according to SAMME implementation, the $\alpha$ determines the contribution of the weak classifier. Here is the Adaboost algorithm $\alpha$ is in step 2. (c) Now in RealAda boost I ...
1
vote
0answers
65 views

Feature selection using correlation

I am trying to do some feature selection using correlation. However, I found that my features are not that correlated. The highest correlation was 0.08. So I am not sure if this is a useful thing to ...
2
votes
0answers
57 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
2
votes
0answers
203 views

Feature importance scores of SVM multiclass one-vs-one design

Info about dataset: 5 classes, 200 trials, 100 features. (I know about the trial to feature ratio being very low, but can not avoid this here and still got well enough classification results.) ...
4
votes
2answers
236 views

How to handle high dimensional feature vector in probability graph model?

I was doing some NLP related stuff which involves training a hidden Markov model, and use the model to segment sentences. For every sentence, I translate the tokens into feature vectors. The features ...
4
votes
1answer
49 views

Which data for feature selection to get unbiased result?

I have a 70 / 30 ratio for train / test data. I have a relatively small feature set (6 features), however, I still want to do feature selection to get rid of any redundant features (I'm guessing 1 of ...
1
vote
0answers
99 views

Event Prediction through Machine Learning

I have a large data set consisting of ca. 40 categorical data items and a few interval data items (real numbers, less than 5 such items). Most categories should have a lot of values that repeat ...
0
votes
1answer
116 views

How to model a multi-dimensional feature set for classification

I am new to statistical modelling and so please pardon if the question appears trivial. I have a set of multi-dimensional data ($T$) where each dimension represents features ($f_i$) obtained from a ...
3
votes
0answers
38 views

How to leverage control / treatment type information in machine learning

I am currently working on an interesting problem that brings together elements of experimental design and machine learning. The setup is as follows. I have $i = 1 \dots N$ subjects. Each subject is ...
1
vote
0answers
29 views

How to use reservoir states for readout and training?

I’m trying to make a Liquid State Machine, I have a spiking neural network as the liquid, and a feedforward neural network that should learn to map the reservoir’s states to the output. I’ve read ...
0
votes
1answer
71 views

How to find important features in this problem

I was just thinking how ML techniques can be applied in the retail industry. Suppose we have data from a retailer who deals with apparel and cloth in this format and for each item there are ...
2
votes
1answer
207 views

Features selection using F-score for multiclass classification

I'm going to implement a feature selection algorithm, and I plan to use the F-score for because of its simplicity. The problem is that, the F-score is used for binary classification. How can it be ...
1
vote
1answer
106 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
2
votes
2answers
725 views

Mutual Information based feature selection

Suppose I have a feature matrix $F = [f_1^T,f_2^T,...,f_m^T]$ whereby $f_j^T \in \mathbb R^{n_t \times 1}$ is the $j$th column vector of $F$ ($n_t$ is the number of different events/trials and $m$ is ...
3
votes
1answer
182 views

Variable importance randomForest negative values

I am asking myself if it is a good idea to remove those variables with a negative variable importance value ("%IncMSE") in a regression context. And if it gives me a better prediction? What do you ...
2
votes
1answer
1k views

What is “feature space”?

What is the definition of "feature space"? For example, When reading about SVMs, I read about "mapping to feature space". When reading about CART, I read about "partitioning to feature space". I ...
1
vote
0answers
120 views

Univariate feature ranking in classification

Scikit-learn has function to evaluate the F-statistics for univariate feature importance feature selection. According to the web page they are calculating ANOVA F value. If I understood correctly, ...
4
votes
2answers
1k views

Number of trees for Random Forest optimization using recursive feature elimination

How many trees would you suggest to pick to perform recursive feature elimination (RFE) in order to optimize Random Forest classifier (for binary classification problem). My dataset is very ...
0
votes
0answers
142 views

What are the good algorithms for feature extraction for large dataset?

I have KDD dataset for detecting fraud actions on networks but it has millions of lines and >20 feature columns. Thus it is not viable to process all these on my personal computer. I am thinking about ...
5
votes
5answers
857 views

Is using the same data for feature selection and cross-validation biased or not?

We have a small dataset (about 250 samples * 100 features) on which we want to build a binary classifier after selecting the best feature subset. Lets say that we partition the data into: Training, ...
5
votes
6answers
519 views

What machine learning algorithms are good for estimating which features are more important?

I have data with a minimum number of features that don't change, and a few additional features that can change and have a big impact on the outcome. My data-set looks like this: Features are A, B, C ...
0
votes
1answer
153 views

R package for feature set algorithm selection

I want to train a binary classification NN and part of this will require data pre-processing. However, I have a choice of which pre-processing algorithm to use. Of course I'd like to choose that one ...