Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
0answers
22 views

Number of candidate inputs that can be handled by different modelling techniques

Am I correct if I say that some modelling techniques can handle better a larger number of candidate inputs? (If we hold the number of observation constant). Let's say I put around 60 different ...
0
votes
0answers
23 views

Should I lower the number of features in linear regression?

I want to do use a a regression package in R on a data which is composed of around 100K samples each with 100K features. Should I do some pre-processing to lower the number of features, or can I ...
0
votes
0answers
61 views

For feature selection: VIF (variance inflation factor) and Correlation Feature Selection with kfold cross validation

Someone asked a question about automatic variable selection for modeling: Algorithms for automatic model selection The community raised concerns that using step() and other methods like that suffer ...
0
votes
0answers
98 views

Caret feature selection [RFE] yields different features depending on reference level of two-class dependent measure

I'm using RFE from the caret package in R to select variables to be used in a linear discriminant analysis. The outcome is a binary factor, but depending on which level of the factor is used as the ...
0
votes
0answers
19 views

Feature Selection for Brain Data

I am trying to make the binary prediction of a certain behavior (present=1, absent=0) from brain activity. I have data from 100 people each with about 40,000 features (regions of brain activity in the ...
0
votes
0answers
27 views

Post-hoc analysis of variable selection

I am using support vector machines & 10-fold cross-validation for a binary classification task. For feature selection, I use the t-test. After doing the classification, I'd like to do a post-hoc ...
3
votes
2answers
95 views

Model Selection in Statistics

I have been told not to look at significance level, or not to use forward/backward selection using BIC/AIC for model selection. Let's say, I have 100 survey data with 11 variables and I want to see ...
0
votes
1answer
33 views

What are the plots I can do in order to select predictors?

First thing I do with the data is to check the relation of the variable to be predicted with other variables. To do this I produce simple plots using plot function or qplot function. I do matrix plot ...
1
vote
0answers
45 views

Feature selection in clustering

I am looking for a method for feature selection in Gaussian Mixture Models. I have a dataset with 2000 records and 40 variables. I tried to use the "clustvarsel" package in R, which use the BIC as ...
0
votes
0answers
23 views

Panel/Longitudinal Data - Seasonality, Variable Selection

I am analyzing a set of panel data by linear regression. I would like to use a fixed effects model, so I am fitting the model below by OLS: $$(y_{it}-\bar y_i)=\beta (X_{it}-\bar X_i) ...
0
votes
1answer
79 views

Best way to determine contribution of a variable to regression model

What is the best way to determine the degree of contribution a variable is making by its addition to a regression model. Suppose I have following regression model for OutNumeric which is a continuous ...
0
votes
0answers
72 views

Variable importance using cforest in clustering / unsupervised learning application

I have a data set which I'd like to cluster by using random forest. As I have more than 50 variables, I first want to identify the most important features and subsequently cluster the data set based ...
4
votes
1answer
36 views

Differences in correlation for individual and aggregated data

I have a sample of 1 million articles form the web with various features. I'm in the progress of selecting features to use in a metric/predictor for article quality. To get some insight into the data ...
0
votes
0answers
11 views

Finding redundant parts of a data set for training a nonlinear model

I like to train a nonlinear model based on a data set $D_1$,..$D_n$, where $D_i$ is collected by doing experiments $E_i$. It is possible that $D_i$ is redundant given $D_j$ and $D_k$. Since doing an ...
1
vote
0answers
19 views

Looking for a principled/systematic procedure for discarding features

I have a collection of $M_i \times N$ matrices $X_i$ whose rows are (raw) feature vectors (from a common $N$-dimensional feature space). MATLAB reports that most of the covariance matrices $C_i := ...
0
votes
2answers
31 views

How to use a varImp function to select features from training set?

Till now I have used a following flow for training a random forest model. ...
0
votes
1answer
58 views

How to compare the relative importance of features in GP regression?

Kernel function with different length scales, such as the squared exponential function, is said to be able to quantify the relative importance among the input (predictor) features. The idea is to ...
0
votes
0answers
21 views

Evaluation of features, how to find which feature is the most effective?

My question is following, which approach should i use in order to make a evaluation of features. To be more specific, for example we have a tweet message: "The weather is nice outside, it makes me ...
1
vote
0answers
53 views

Including Feature Selection in Cross Validation - Application to Bag of Words

I am working on a prediction problem where I was given a 6,000 record dataset with the value of the dependent variable included ("train"), and a 2,000 record dataset with the same independent ...
1
vote
0answers
15 views

Feature Extraction for landscape images

I work on a landscape-image database (forests, beaches, landmarks, cities etc.) and I'm trying to find the similarity between them, I don't have labels or any specific classes, I just want to find ...
0
votes
0answers
24 views

Which one is correct phase for neural network or support vector machine? Features or Inputs?

Which one is correct phase for Neural Network or Support Vector Machine? Features or Inputs? Based on ...
2
votes
1answer
145 views

Features that correspond to rare events: how rare is “too rare” to be informative?

I am working with 82 binary features constructed from six categorical features. I have about 1,600 observations. Some of these features correspond to extremely rare categories. Some of them have only ...
0
votes
0answers
40 views

Tree analysis - CHAID cart

I am new to CHAID and want to know how to decide which independent variables I should select to run CHAID Analyis? Is there a technique to select and then apply them and run the analysis? Please guide ...
1
vote
1answer
55 views

Can adding an additional feature to a perceptron classifier make the results worse?

I am using perceptron to solve a classification problem. I have a limited amount of features (26) and iterate through all possible combinations of them. A combination of two features [feature_a, ...
0
votes
0answers
29 views

What will be the simple interpretation for the coefficients for features obtained in any Machine learning models?

I am working with a data that consists of two classes. I have used scikit learn, to craete models using SVM, Randomforest etc.I used to r2_score and I sorted the scores for features I am having and I ...
2
votes
0answers
49 views

How to make use of less data of a particular class for better modeling?

I have a dataset, say 9000 rows, with some features. Around 8000 belong to class 1 and 1000 to class 0. So, if I am creating a model with any method say SVM, LR, Random forest the model has a tendency ...
0
votes
0answers
34 views

Feature Representation for Samples with Different Number of Properties

I want to build a machine-learning classification model that learn from properties (features) extracted from proteins. I represent each sample (protein) using some features (e.g. 100 features ...
1
vote
1answer
75 views

Scale-invariant feature transform explanation

How do I explain the scale-invariant feature transform (SIFT) to a layman?
0
votes
0answers
20 views

I observed very different feature scoring from two different classifiers. What does it really mean?

Here what I've done. Given the dataset, I run a Random Forests and Logistic Regression with 5 Fold Stratified Data Sampling. Then I plot the feature importance for Random Forests and Logistic ...
2
votes
1answer
41 views

Automatically fixing ill-conditioning or collinearity

I'm backtesting a regression model, which entails running it on a bunch of bootstrap samples of a "rewound" version of our data set. Unfortunately, in some of these resamplings, I end up getting some ...
0
votes
0answers
27 views

Appending additional data to learnt autoencoder features

My task is to perform image segmentation / full scene image parsing. I am working on an outdoor dataset which was taken under strict spatial constraints. The images contain fruit on trees and the ...
1
vote
1answer
124 views

No significant tests when using Benjamini-Yekutieli multiple testing correction on millions of tests

I am using a univariate filter to reduce the number of features prior to applying a learning algorithm to a huge binary classification dataset (22510066 features x 500 examples). All the features are ...
2
votes
0answers
103 views

Deep learning: representation learning or classification?

For classification, I have often heard about deep learning / deep neural networks as a form of representation learning. I am confused as to what "representation learning" means in this context. Which ...
2
votes
0answers
49 views

Reference for this claim: important features in data can be “hidden” in the higher PCA axes that are typically thrown out [duplicate]

I remember reading a paper a while ago that demonstrated some cases in which PCA would fail to capture important features of a data set in the first few principal components, but where those features ...
0
votes
2answers
27 views

R feature selection [duplicate]

I am working with the method randomForest for model building. And for a good model performance, it is very important to select the right features. At my example I have 30 variables and I would like to ...
0
votes
0answers
12 views

How can be assesed that a given data representation is better than the other?

Given a classification dataset, suppose I learn many different data representation with Matrix Factorization, Clustering or with such approaches. At the end , how would I decide which is better than ...
0
votes
0answers
31 views

Data-driven, high-dimensional feature selection strategies

I am working on a biomedical/healthcare data science problem. I have a dataset of 600 samples, ~6000 variables and class label as "positive" or "negative". I want to perform feature selection on ...
0
votes
1answer
57 views

Classifier with variable number of features

I am trying to make a classifier when each sample has a variable number of features. An example of how this could occur is, for example, if the features are the purchases (type, dollar amount, etc) ...
1
vote
2answers
111 views

How To Better Represent A Problem To A Machine Learning Algorithm

I am familiar with the basics of how to present a problem to a machine learning algorithm using binary encodings. I am also familiar with, but still learning about, feature selection/extraction and ...
0
votes
0answers
28 views

Find the relative importance of features weights in multi-class SVM without PCA - plotting coef distribution?

I'm classifing users with a multiclass svm (one-against-on), 3 classes. In binary, I would be able to plot the distribution of the weight of each feature in the hyperplan equation for different ...
0
votes
0answers
32 views

feature slection in random forest in python

I have a dataset consisting of 24 numeric features and about 7000 rows, i am applying random forest to get the binary classification, So please tell me how to find only the relevant features to get ...
0
votes
0answers
12 views

What's the probability that there exists a hyperplane that can split a dataset which have random feature values ?

Given n data points, each with d features, n/2 are labeled as -1, the other n/2 are labeled as 1. Each feature takes a value from [0,1] randomly (uniform distribution). What's the probability that ...
1
vote
1answer
145 views

Justification for feature selection by removing predictors with near zero variance

I have a large number of variables that I'm trying to reduce, and I've stumbled on Kuhn's (2008) suggestion that I eliminate variables with zero or near-zero variance. This makes sense to me, it's ...
2
votes
1answer
61 views

Will adding additional features hurt the performance of SVM ?

Just wondering the effects of additional features. Following are several thoughts: If the additional features are noisy (can not distinguish the two classes), then additional features won't hurt ...
1
vote
1answer
27 views

What kind of general strategy can you apply after selecting model and hyper parameter training?

As a rookie to machine learning area, I tried to play some Data Science tutorials and beginner competitions to gain some knowledge and experience. The problem I encountered in every scenarios is ...
1
vote
0answers
20 views

Relation between chi-squared statistic scores and classification accuracy

I am evaluating the utility of two distinct sets of features for solving a given supervised classification problem with two classes. I am using the chi-squared statistic as a feature selection ...
0
votes
0answers
17 views

Finding the features that have meaning in subset of data

I have a set of $N$ points $x_i=(x_i^1, x_i^2,...,x_i^{m+k})$ in $m+k$-dimensional space ($m$ continuous dimensions and $k$ discrete). Also I have a subset of these points that are marked as "bad". ...
0
votes
1answer
67 views

After adding additional features, same accuracy on test data, but higher accuracy on training data, how should I interpret ?

I've done 5-fold cross-validation and the model is SVM. 300 features: 0.53 on test, 0.55 on training; 700 featuers: 0.53 on test, 0.67 on training. Does this mean that the additional 400 features ...
2
votes
1answer
239 views

Feature selection for time series data

I am looking for methods for feature selection (or feature extraction) for time series data. Of course I did some research before, but it was not satisfying. I am aware of methods like PCA, ...
1
vote
0answers
32 views

High dimensional explanatory variable

I have a data set of 22 observations and 6931 variables. Data belongs to two classes, 0 and 1. I would like to know which features are important for each class (species) and which one contribute the ...