Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
0answers
11 views

Including Feature Selection in Cross Validation - Application to Bag of Words

I am working on a prediction problem where I was given a 6,000 record dataset with the value of the dependent variable included ("train"), and a 2,000 record dataset with the same independent ...
0
votes
1answer
369 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
1
vote
0answers
8 views

Feature Extraction for landscape images

I work on a landscape-image database (forests, beaches, landmarks, cities etc.) and I'm trying to find the similarity between them, I don't have labels or any specific classes, I just want to find ...
1
vote
2answers
135 views

Extract important features

Here is my situation: - A huge amount of data - 600 features - Only one class is provided Now, my question is how can I reduce the number of features to important ones? In another word, all of these ...
0
votes
0answers
8 views

Datasets with a specific decision structure [on hold]

I'm looking for classification datasets where the prediction variable has a certain structure, i.e. if the possible prediction outcomes are natural numbers between 1 and 10, the user is more preferred ...
0
votes
0answers
18 views

Which one is correct phase for neural network or support vector machine? Features or Inputs?

Which one is correct phase for Neural Network or Support Vector Machine? Features or Inputs? Based on ...
1
vote
0answers
35 views

giving some predictors higher priority in statistical models

i am working on statistical software debugging. i use various feature selection algorithms in order to discover the bug predictors in programs. for example regression coefficients indicate the ...
2
votes
1answer
106 views

Features that correspond to rare events: how rare is “too rare” to be informative?

I am working with 82 binary features constructed from six categorical features. I have about 1,600 observations. Some of these features correspond to extremely rare categories. Some of them have only ...
0
votes
0answers
18 views

Tree analysis - CHAID cart

I am new to CHAID and want to know how to decide which independent variables I should select to run CHAID Analyis? Is there a technique to select and then apply them and run the analysis? Please guide ...
0
votes
0answers
23 views
1
vote
1answer
30 views

Can adding an additional feature to a perceptron classifier make the results worse?

I am using perceptron to solve a classification problem. I have a limited amount of features (26) and iterate through all possible combinations of them. A combination of two features [feature_a, ...
0
votes
0answers
21 views

What will be the simple interpretation for the coefficients for features obtained in any Machine learning models?

I am working with a data that consists of two classes. I have used scikit learn, to craete models using SVM, Randomforest etc.I used to r2_score and I sorted the scores for features I am having and I ...
2
votes
0answers
34 views

How to make use of less data of a particular class for better modeling?

I have a dataset, say 9000 rows, with some features. Around 8000 belong to class 1 and 1000 to class 0. So, if I am creating a model with any method say SVM, LR, Random forest the model has a tendency ...
1
vote
1answer
49 views
0
votes
0answers
25 views

Feature Representation for Samples with Different Number of Properties

I want to build a machine-learning classification model that learn from properties (features) extracted from proteins. I represent each sample (protein) using some features (e.g. 100 features ...
0
votes
0answers
18 views

I observed very different feature scoring from two different classifiers. What does it really mean?

Here what I've done. Given the dataset, I run a Random Forests and Logistic Regression with 5 Fold Stratified Data Sampling. Then I plot the feature importance for Random Forests and Logistic ...
2
votes
1answer
31 views

Automatically fixing ill-conditioning or collinearity

I'm backtesting a regression model, which entails running it on a bunch of bootstrap samples of a "rewound" version of our data set. Unfortunately, in some of these resamplings, I end up getting some ...
0
votes
0answers
21 views

Appending additional data to learnt autoencoder features

My task is to perform image segmentation / full scene image parsing. I am working on an outdoor dataset which was taken under strict spatial constraints. The images contain fruit on trees and the ...
1
vote
2answers
183 views

Evaluating features and similarity measures

I am currently developing a classifier, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
1
vote
1answer
60 views

No significant tests when using Benjamini-Yekutieli multiple testing correction on millions of tests

I am using a univariate filter to reduce the number of features prior to applying a learning algorithm to a huge binary classification dataset (22510066 features x 500 examples). All the features are ...
-1
votes
0answers
25 views

Clustering binary letters/text - image features and descriptors

I have a task to cluster 1 000 000 images (where each image represents one letter). They are all in binary format. I need to get clusters where each of them would represent 1 letter.. Which image ...
2
votes
0answers
47 views

Reference for this claim: important features in data can be “hidden” in the higher PCA axes that are typically thrown out [duplicate]

I remember reading a paper a while ago that demonstrated some cases in which PCA would fail to capture important features of a data set in the first few principal components, but where those features ...
8
votes
4answers
188 views

How can top $k$ principal components retain the predictive power on a dependent variable?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from ...
2
votes
0answers
50 views

Deep learning: representation learning or classification?

For classification, I have often heard about deep learning / deep neural networks as a form of representation learning. I am confused as to what "representation learning" means in this context. Which ...
0
votes
1answer
65 views

combining multiple classifiers common features

Can multiple binary-classifiers be combined to produce a final output if their feature sets have some common elements? How will this influence the accuracy?
7
votes
1answer
1k views

Clustering probability distributions - methods & metrics?

I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
2
votes
1answer
419 views

Random forest cross validation for feature selection, imbalanced datasets

I have an 5297X26 imbalanced dataset, the class1 has 588 samples and class2 has 4709 samples. I used the following code to perform random forest: ...
1
vote
1answer
98 views

Feature selection for pattern mining

I must find frequent patterns in temporal data, using a method that was imposed to me. This tool has problems handling these data: processing is long and takes a lot of memory. So, I would like to ...
2
votes
1answer
176 views

Determining conserved features using a Bayesian approach

I would like to perform some sort of binary classification, and my data set consists of 100 examples (for each class), which are vectors with 2500 elements. Ideally, I would like to determine which ...
0
votes
0answers
6 views

How can be assesed that a given data representation is better than the other?

Given a classification dataset, suppose I learn many different data representation with Matrix Factorization, Clustering or with such approaches. At the end , how would I decide which is better than ...
0
votes
1answer
24 views

Classifier with variable number of features

I am trying to make a classifier when each sample has a variable number of features. An example of how this could occur is, for example, if the features are the purchases (type, dollar amount, etc) ...
1
vote
2answers
63 views

How To Better Represent A Problem To A Machine Learning Algorithm

I am familiar with the basics of how to present a problem to a machine learning algorithm using binary encodings. I am also familiar with, but still learning about, feature selection/extraction and ...
0
votes
0answers
22 views

Data-driven, high-dimensional feature selection strategies

I am working on a biomedical/healthcare data science problem. I have a dataset of 600 samples, ~6000 variables and class label as "positive" or "negative". I want to perform feature selection on ...
0
votes
0answers
8 views

Find the relative importance of features weights in multi-class SVM without PCA - plotting coef distribution?

I'm classifing users with a multiclass svm (one-against-on), 3 classes. In binary, I would be able to plot the distribution of the weight of each feature in the hyperplan equation for different ...
0
votes
0answers
14 views

feature slection in random forest in python

I have a dataset consisting of 24 numeric features and about 7000 rows, i am applying random forest to get the binary classification, So please tell me how to find only the relevant features to get ...
1
vote
1answer
15 views

What kind of general strategy can you apply after selecting model and hyper parameter training?

As a rookie to machine learning area, I tried to play some Data Science tutorials and beginner competitions to gain some knowledge and experience. The problem I encountered in every scenarios is ...
0
votes
0answers
11 views

What's the probability that there exists a hyperplane that can split a dataset which have random feature values ?

Given n data points, each with d features, n/2 are labeled as -1, the other n/2 are labeled as 1. Each feature takes a value from [0,1] randomly (uniform distribution). What's the probability that ...
2
votes
1answer
35 views

Will adding additional features hurt the performance of SVM ?

Just wondering the effects of additional features. Following are several thoughts: If the additional features are noisy (can not distinguish the two classes), then additional features won't hurt ...
0
votes
1answer
152 views

Which feature selection method to use for classification problem

I have to do some feature selection for a classification problem with numeric features. I am not sure which feature selection method to use. Chisquared test or Spearmann's rank correlation ...
1
vote
1answer
231 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
1
vote
2answers
92 views

Feature selection before neural network classification

I have a training set of 87 samples and 9480 variables. My predictors are continuous and my response variable is binary. I'd like to use the caret package in R to tune a neural network classification ...
0
votes
1answer
25 views

After adding additional features, same accuracy on test data, but higher accuracy on training data, how should I interpret ?

I've done 5-fold cross-validation and the model is SVM. 300 features: 0.53 on test, 0.55 on training; 700 featuers: 0.53 on test, 0.67 on training. Does this mean that the additional 400 features ...
1
vote
0answers
13 views

Relation between chi-squared statistic scores and classification accuracy

I am evaluating the utility of two distinct sets of features for solving a given supervised classification problem with two classes. I am using the chi-squared statistic as a feature selection ...
0
votes
0answers
9 views

Finding the features that have meaning in subset of data

I have a set of $N$ points $x_i=(x_i^1, x_i^2,...,x_i^{m+k})$ in $m+k$-dimensional space ($m$ continuous dimensions and $k$ discrete). Also I have a subset of these points that are marked as "bad". ...
1
vote
1answer
52 views

Feature selection for time series data

I am looking for methods for feature selection (or feature extraction) for time series data. Of course I did some research before, but it was not satisfying. I am aware of methods like PCA, ...
5
votes
1answer
1k views

Gini decrease and Gini impurity of children nodes

I'm working on the Gini feature importance measure for random forest. Therefore, I need to calculate the Gini decrease in node impurity. Here is the way I do so, which leads to a conflict with the ...
1
vote
0answers
25 views

High dimensional explanatory variable

I have a data set of 22 observations and 6931 variables. Data belongs to two classes, 0 and 1. I would like to know which features are important for each class (species) and which one contribute the ...
2
votes
1answer
110 views

Feature selection and training on the same sample

Is feature selection and training on the same sample a bad idea? I want to emphasize that I am not going to use test set for feature selection. If I use the whole train set for feature selection and ...
1
vote
0answers
21 views

Learning if instances from a dataset are part of the same subset

I was wondering if there are some well-known machine learning methodologies for subset learning. In other words, to learn if two instances are part of the same subset or not (boolean label?). One ...