Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

0
votes
1answer
104 views

Grid Search for hyperparameter and feature selection

So I need to select my hyperparameters and also my features. A full grid search of the space of hyperparameters and features is too computationally intensive, so what I am doing instead is for each ...
1
vote
0answers
40 views

How to determine appropriate number of features and also which features to select?

So I have a dataset which I am using K fold cross validation on to select the number of features and which features should be selected. As I understand it, I would set the number of features to be ...
1
vote
0answers
40 views

What are some classic examples of feature selection in classification?

Is there a classic example showing the importance of good feature selection in classification? The ideal example would be simple, and very easy to understand. I've been volunteered/instructed to put ...
2
votes
1answer
335 views

The main effect will be non-significant if the interaction is significant? [duplicate]

I am using linear mixed models to identify important factors, and it turns out that: A: significant B: not significant ...
0
votes
1answer
73 views

What is parameter fine tuning means in SVM?

I got this sentence in one of paper, but I dont understand what does it mean?? "Training a learningbased classifier such as an SVM on an imbalanced dataset often requires parameter fine-tuning, ...
0
votes
2answers
37 views

How to determine the factors correlated with observed data?

I have box-office collection data on a number of movies. I also have the production budget, director name, lead actor, actress, language and other meta data related to the movie. I want to know which ...
2
votes
1answer
69 views

In decision tree construction, can a good splitter have low information gain?

I have a data set with a candidate splitter variable that is a natural choice from the business perspective. It has two values, and the distributions of the target when conditioned on the two values ...
4
votes
3answers
619 views

Can independent variables with low correlation with dependent variable be significant predictors?

I have eight independent variables and one dependent. I have run a correlation matrix, and 5 of them have a low correlation with the DV. I have then run a stepwise multiple regression to see whether ...
3
votes
1answer
94 views

SVM basic theory?

I have some questions about SVM: In SVM there is a nonlinear and linear SVM. What is the difference between them? To do classification in SVM, we will find the linearly separable boundary ...
0
votes
0answers
14 views

Variable Selection Methods in R [duplicate]

regsubsets and stepAIC are the two most common options for variable selection in R; they can be found in the ...
0
votes
0answers
27 views

Detecting noisy patterns in document images

I am looking for features to extract to distinguish between text objects and arbitrary noisy patterns in degraded document images. This is an example of a document with some parts of noise, I have ...
0
votes
0answers
11 views

Error trying to reduce my data dimentions [duplicate]

I'm trying to produce a linear regression model, but I only have 25 observations and 34 predictors. I'm trying feature selection, ...
0
votes
1answer
108 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
1
vote
0answers
19 views

How to extract “average polyline” from a set of polylines [duplicate]

EDIT: As pointed, there is already a similar question "buried" inside a larger-scope one. I'll reproduce the relevant part here: If I have multiple recordings of the same routes are there any ...
1
vote
1answer
65 views

Feature selection for pattern mining

I must find frequent patterns in temporal data, using a method that was imposed to me. This tool has problems handling these data: processing is long and takes a lot of memory. So, I would like to ...
0
votes
0answers
20 views

Segmenting an interval sensibly

Is there a canonical/recommended approach to or algorithm for splitting up an interval with the intent of minimizing the number of segments while keeping a high accuracy? It is essentially an ...
0
votes
1answer
144 views

Good algorithms for feature extraction from images?

I am searching for some algorithms for feature extraction from images which I want to classify using machine learning . I have heard only about [scale-invariant feature transform][1] (SIFT), I have ...
1
vote
0answers
48 views

Why do Laplacian Eigenmaps ignore the second KKT condition in the solution calculation?

From the original Laplacian Eigenmaps paper, we find the new manifold embedding $Y$ as follows, given the Laplacian and degree matrices $L$ and $D$: $\arg\min_Ytr(Y^TLY)$ subject to ${Y^TDY=I}$. ...
1
vote
0answers
24 views

Clustering and Feature Selection For Audio Data

I'm very new to stats and I'm at a loss as to where to start with this problem. I'm not sure what tools or methods to use to extract meaningful answers from my data. I'll try and describe the problem ...
0
votes
2answers
44 views

Selecting features and estimating their out-of-sample performance with cross-validation

I have only a small dataset. I want to 1. select the most predictive features out of a large candidate pool and 2. get an estimate of their expected predictive performance. In the elements of ...
2
votes
0answers
41 views

Quantify the information lost given by the Kullback-Leibler divergence measure

Consider there are $N$ individuals and these measure a quantity $X\in \mathbb{R}^{N\times M}$ where $M$ is the number of measurements and let $P(X)$ denote a probability distribution over $X$. The ...
2
votes
1answer
36 views

Term selection in a model

In selecting terms to include in a model, say a linear one, should we always test the significance of the main effects first, keep only the significant ones, then consider the possible interactions ...
0
votes
0answers
48 views

Recursive feature extraction with random forests following by statistical inference?

I have a small dataset of n = 200 with about 80 predictors. The outcome I am working with is binary. I'm interested in doing a two-step process where: (1) I conducted recursive feature extraction ...
3
votes
2answers
336 views

Cross-validation and feature selection of a multivariate regression

I've been trying to create a multivariate regression model to fit my training data into the prediction of a value. I've put my data into a matrix X with ...
1
vote
0answers
39 views

Linear regression for feature selection

Imagine we regress y on x1...x4. Now, we want to find out if ...
3
votes
2answers
73 views

Advice for interpolating a model

I'm new in Stack Exchange, so I hope no to be off topic. I'm also new in bioinformatics and I was asked to perform an analysis. Briefly, I have a dataset of 29 cell lines and the IC50 values of a test ...
3
votes
1answer
104 views

Interpreting the lasso coefficients

I have used lasso logistic regression on some data and I have some non zero coefficients for some of the features. I want to know based upon the coefficient values how do I rank the features?
0
votes
0answers
45 views

Feature selection for support vector regression with time series as features

I would like to select features for a support vector regression for forecasting. I would like to forecast a value at point t with the values t-1,...t-x as features. Now I want to select the most ...
0
votes
1answer
44 views

How to compare features and classifiers which achieve perfect accuracy?

So I'm looking to compare different combinations of features and classifiers. But I'm getting a lot of combinations that achieve 100% cross validation accuracy. I'm trying to figure out how I would ...
0
votes
0answers
55 views

Is it possible to determine the set of variables contributing the most to first two principal components? [duplicate]

TL;DR: Is it possible to determine the set of features (covariates) that contribute the most to the first two principal components, and if so, how? Long version: Let's say I have a data table where ...
3
votes
3answers
42 views

How to compare features and classifiers which achieve perfect accuracy?

So I'm looking to compare different combinations of features and classifiers. But I'm getting a lot of combinations that achieve 100% cross validation accuracy. I'm trying to figure out how I would ...
2
votes
1answer
46 views

Feature selection while retaining a specified feature

Pardon if this question is very basic, but I am not able to find any solution for my problem. I am trying to run a feature selection scheme on N features for my classification model, however I want ...
0
votes
0answers
30 views

Proper way to determine attribute feature selection's smaller subset based on result metrics

Overview My goal is to predict survival of an instance for five different time periods (binary attribute). I have a 100,000-instance dataset with 40 attributes and I want to reduce the attributes ...
4
votes
1answer
123 views

Variable selection for regression - the subselect package

No regular here will be unaware of the perils of using stepwise and similar automatic methods for variable selection in regression analysis. But preferred alternatives, such as the lasso or ...
1
vote
0answers
50 views

Music classification into genres

I need to classify music (songs) into genres (rock, french house, trash metal, etc). My idea was to extract features from the songs (bmp, zero crossing, etc) and then apply known classification ...
0
votes
0answers
87 views

Recursive feature elimination with only two classes

Recursive feature elimination (RFE) is a feature-selection strategy. It performs in two nested levels of cross-validation. First it tries to divide the training set into N folds. RFE puts one fold ...
1
vote
0answers
60 views

Classifier predicts only one class

I was trying myself in kaggle CIFAR competition, I trained lots of classifiers but get the same result/fail (don't know how to treat them), maybe someone could help me figure what i'm doing wrong. ...
0
votes
0answers
19 views

How to interpret the selection of a variable in a model (GLM) when spatialautocorrelation is included?

This is a pretty straightforward question. I am comparing outputs of two models (binomial GLM) one including environnmental-only (ENV) variables and one including environmental and spatial variables ...
3
votes
1answer
559 views

Understanding the output of C5.0 classification model using the CARET package

The C5.0 classification model was used in this 4-class problem data with $N_{train}$=165, $P$=11, using caret R-package by ...
1
vote
0answers
29 views

Estimating confidence of a prediction

Given a set of features vectors $X=\{\vec{x}_1,..,\vec{x}_n\}$, binary ground truth data $Y=\{y_1,..,y_n\}$ and continuous prediction $\bar{Y} = \{\bar{y}_1,..,\bar{y}_n\}\in [0,1]$, I want to perform ...
1
vote
0answers
67 views

Feature selection methods comparison

I recently run a project that involves a feature selection step before further pattern recognition. The number of features for our data set is very large and instead of running greedy ...
1
vote
2answers
138 views

The curse of dimensionality? (linear SVMs)

How do you know whether you suffer from it? Let's suppose I have a 2 class problem - 2000 training examples and 30 features. While it works good for the most part, sometimes I get edge cases that ...
1
vote
1answer
39 views

Algorithms/methods to create more features of a limited amount of features?

So, let's suppose that I have a set of 20 features - some of them are continous and some of them are binary. Is there an algorithm/method/solution to create more features ( combine/transform ) those ...
2
votes
1answer
208 views

How to interpret this cross-validated sparse LDA figure using CARET package?

Training data with $p$ =11 predictors and $n$ =165 with 4-class problem was cross-validated (5 times repeated 10-fold CV) using the sparse LDA (aka SDA) using caret ...
1
vote
1answer
39 views

SVM - combining binary and continous representation of the same feature?

How would this influence the accuracy of the SVM model? Let's suppose that I have one variable which max value is 100 and minimum is 0. Currently, I send it to SVM as a single continuous feature, ...
1
vote
1answer
27 views

Significance Test for Comparison of Variables

I am not sure how to ask this question without giving an example. I am trying to measure the "cleanliness" of office buildings. I have two variables that try to measure this. Variable one is a ...
1
vote
1answer
106 views

Using Principal Components Analysis for feature selection

I have a dataset D made of m samples, and n features with n >> m. For each sample I have a score s which I would like to ...
1
vote
2answers
307 views

Mixing continuous and binary data with linear SVM?

So I've been playing around with SVMs and I wonder if this is a good thing to do: I have a set of continuous features (0 to 1) and a set of categorical features that I converted to dummy variables. ...
2
votes
2answers
153 views

Selecting a feature modeling approach for text classification

I am new to text processing. Currently I am trying to determine which type of feature vector I need for a classification problem. I am mainly deciding between binary feature modeling and ...
0
votes
0answers
18 views

Neural net: variable distribution, validation and number of variables - unexpected results

I have been experimenting with ANNs on one of my datasets, they seem to have the potential to be quite effective in explaining my Y variation. Something i am finding is that they very much benefit ...