Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

1
vote
0answers
50 views

Why do Laplacian Eigenmaps ignore the second KKT condition in the solution calculation?

From the original Laplacian Eigenmaps paper, we find the new manifold embedding $Y$ as follows, given the Laplacian and degree matrices $L$ and $D$: $\arg\min_Ytr(Y^TLY)$ subject to ${Y^TDY=I}$. ...
1
vote
0answers
28 views

Clustering and Feature Selection For Audio Data

I'm very new to stats and I'm at a loss as to where to start with this problem. I'm not sure what tools or methods to use to extract meaningful answers from my data. I'll try and describe the problem ...
0
votes
2answers
52 views

Selecting features and estimating their out-of-sample performance with cross-validation

I have only a small dataset. I want to 1. select the most predictive features out of a large candidate pool and 2. get an estimate of their expected predictive performance. In the elements of ...
2
votes
0answers
53 views

Quantify the information lost given by the Kullback-Leibler divergence measure

Consider there are $N$ individuals and these measure a quantity $X\in \mathbb{R}^{N\times M}$ where $M$ is the number of measurements and let $P(X)$ denote a probability distribution over $X$. The ...
2
votes
1answer
36 views

Term selection in a model

In selecting terms to include in a model, say a linear one, should we always test the significance of the main effects first, keep only the significant ones, then consider the possible interactions ...
0
votes
0answers
62 views

Recursive feature extraction with random forests following by statistical inference?

I have a small dataset of n = 200 with about 80 predictors. The outcome I am working with is binary. I'm interested in doing a two-step process where: (1) I conducted recursive feature extraction ...
4
votes
2answers
432 views

Cross-validation and feature selection of a multivariate regression

I've been trying to create a multivariate regression model to fit my training data into the prediction of a value. I've put my data into a matrix X with ...
1
vote
0answers
47 views

Linear regression for feature selection

Imagine we regress y on x1...x4. Now, we want to find out if ...
3
votes
2answers
77 views

Advice for interpolating a model

I'm new in Stack Exchange, so I hope no to be off topic. I'm also new in bioinformatics and I was asked to perform an analysis. Briefly, I have a dataset of 29 cell lines and the IC50 values of a test ...
3
votes
1answer
152 views

Interpreting the lasso coefficients

I have used lasso logistic regression on some data and I have some non zero coefficients for some of the features. I want to know based upon the coefficient values how do I rank the features?
0
votes
0answers
49 views

Feature selection for support vector regression with time series as features

I would like to select features for a support vector regression for forecasting. I would like to forecast a value at point t with the values t-1,...t-x as features. Now I want to select the most ...
0
votes
1answer
46 views

How to compare features and classifiers which achieve perfect accuracy?

So I'm looking to compare different combinations of features and classifiers. But I'm getting a lot of combinations that achieve 100% cross validation accuracy. I'm trying to figure out how I would ...
3
votes
3answers
54 views

How to compare features and classifiers which achieve perfect accuracy?

So I'm looking to compare different combinations of features and classifiers. But I'm getting a lot of combinations that achieve 100% cross validation accuracy. I'm trying to figure out how I would ...
2
votes
1answer
47 views

Feature selection while retaining a specified feature

Pardon if this question is very basic, but I am not able to find any solution for my problem. I am trying to run a feature selection scheme on N features for my classification model, however I want ...
0
votes
0answers
35 views

Proper way to determine attribute feature selection's smaller subset based on result metrics

Overview My goal is to predict survival of an instance for five different time periods (binary attribute). I have a 100,000-instance dataset with 40 attributes and I want to reduce the attributes ...
4
votes
1answer
157 views

Variable selection for regression - the subselect package

No regular here will be unaware of the perils of using stepwise and similar automatic methods for variable selection in regression analysis. But preferred alternatives, such as the lasso or ...
1
vote
0answers
57 views

Music classification into genres

I need to classify music (songs) into genres (rock, french house, trash metal, etc). My idea was to extract features from the songs (bmp, zero crossing, etc) and then apply known classification ...
0
votes
1answer
144 views

Recursive feature elimination with only two classes

Recursive feature elimination (RFE) is a feature-selection strategy. It performs in two nested levels of cross-validation. First it tries to divide the training set into N folds. RFE puts one fold ...
1
vote
0answers
69 views

Classifier predicts only one class

I was trying myself in kaggle CIFAR competition, I trained lots of classifiers but get the same result/fail (don't know how to treat them), maybe someone could help me figure what i'm doing wrong. ...
0
votes
0answers
21 views

How to interpret the selection of a variable in a model (GLM) when spatialautocorrelation is included?

This is a pretty straightforward question. I am comparing outputs of two models (binomial GLM) one including environnmental-only (ENV) variables and one including environmental and spatial variables ...
3
votes
1answer
835 views

Understanding the output of C5.0 classification model using the CARET package

The C5.0 classification model was used in this 4-class problem data with $N_{train}$=165, $P$=11, using caret R-package by ...
1
vote
0answers
30 views

Estimating confidence of a prediction

Given a set of features vectors $X=\{\vec{x}_1,..,\vec{x}_n\}$, binary ground truth data $Y=\{y_1,..,y_n\}$ and continuous prediction $\bar{Y} = \{\bar{y}_1,..,\bar{y}_n\}\in [0,1]$, I want to perform ...
1
vote
0answers
95 views

Feature selection methods comparison

I recently run a project that involves a feature selection step before further pattern recognition. The number of features for our data set is very large and instead of running greedy ...
1
vote
2answers
199 views

The curse of dimensionality? (linear SVMs)

How do you know whether you suffer from it? Let's suppose I have a 2 class problem - 2000 training examples and 30 features. While it works good for the most part, sometimes I get edge cases that ...
1
vote
1answer
44 views

Algorithms/methods to create more features of a limited amount of features?

So, let's suppose that I have a set of 20 features - some of them are continous and some of them are binary. Is there an algorithm/method/solution to create more features ( combine/transform ) those ...
2
votes
1answer
282 views

How to interpret this cross-validated sparse LDA figure using CARET package?

Training data with $p$ =11 predictors and $n$ =165 with 4-class problem was cross-validated (5 times repeated 10-fold CV) using the sparse LDA (aka SDA) using caret ...
1
vote
1answer
41 views

SVM - combining binary and continous representation of the same feature?

How would this influence the accuracy of the SVM model? Let's suppose that I have one variable which max value is 100 and minimum is 0. Currently, I send it to SVM as a single continuous feature, ...
1
vote
1answer
28 views

Significance Test for Comparison of Variables

I am not sure how to ask this question without giving an example. I am trying to measure the "cleanliness" of office buildings. I have two variables that try to measure this. Variable one is a ...
1
vote
1answer
156 views

Using principal component analysis (PCA) for feature selection in regression [duplicate]

I have a dataset $D$ made of $m$ samples and $n$ features with $n \gg m$. For each sample I have a score $s$ which I would like to be able to predict. As the number of features is very high (compared ...
1
vote
2answers
476 views

Mixing continuous and binary data with linear SVM?

So I've been playing around with SVMs and I wonder if this is a good thing to do: I have a set of continuous features (0 to 1) and a set of categorical features that I converted to dummy variables. ...
2
votes
2answers
175 views

Selecting a feature modeling approach for text classification

I am new to text processing. Currently I am trying to determine which type of feature vector I need for a classification problem. I am mainly deciding between binary feature modeling and ...
0
votes
0answers
20 views

Neural net: variable distribution, validation and number of variables - unexpected results

I have been experimenting with ANNs on one of my datasets, they seem to have the potential to be quite effective in explaining my Y variation. Something i am finding is that they very much benefit ...
2
votes
1answer
103 views

Are there any techniques that quantify the importance/signification of individual attribute values of a particular data point?

Are there any techniques that quantize the importance of individual attribute values in a particular data point, in terms of the attribute's overall importance/signification/contribution to the ...
0
votes
0answers
83 views

equivalent of PCA explained variance ratio for SVD ?

i am wondering if there is an equivalent of PCA explained variance ratio for SVD. What are the measures I can get to monitor the number of columns I keep after the SVD ? Are any of these metrics ...
0
votes
0answers
27 views

Investigating 'minor' effect variables?

Is there any way to investigate minor contributing $X$ variables in a model when there are one or two $X$ variables which contribute to the explanation of a majority of the variation in the $Y$ ...
5
votes
1answer
185 views

Explain steps of LLE (local linear embedding) algorithm?

I understand the basic principle behind the algorithm for LLE consists of three steps. Finding the neighborhood of each data point by some metric such as k-nn. Find weights for each neighbor which ...
0
votes
0answers
50 views

Feature Selection: markov blanket filter

I need to do a markov blanket filter for feature selection for highly unbalanced datasets. There are popular algorithms to do this? I need to understand the algorithm behind this. From what I ...
0
votes
0answers
25 views

Choice of 0 or -1 for failure in the independent variables of a logistic regression

I am performing some exploratory analysis on a dataset where the dependent variable is a dichotomous variable. I have ~10 explanatory variables, some of which are dichotomous observations. I am ...
0
votes
2answers
81 views

Choosing the best featureset for prediction

I have this N sets of features F each with $F_i$ number of features. All the feature sets have 20000 examples and we have 20,000 labels for them. Lets say feature set $F_1$ has 10 features and ...
2
votes
1answer
107 views

How to calculate number of features based on image resolution?

Just covered Andrew Ng's Non-linear Hypothesis of Neural Netowrks, and we had a multiple choice question for determining number of features for an image of resolution 100x100 of grescale intensities. ...
2
votes
2answers
127 views

improve precision in text classification

I am working on binary text classification using sklearn: The length of each sample is not high (~ 200-500 characters) I use TF-IDF to get important words as TfidfVectorizer(sublinear_tf=False, ...
2
votes
1answer
90 views

Purposeful selection and confounding

I conducted purposeful selection as outlined in Jewell's Statistics for Epidemiology. The log likelihood tests showed covariates, which I considered to be confounding though not significant in the ...
1
vote
0answers
51 views

Inferring dimension weight in a mapping from a triangle to a distribution over its vertices

I have a dataset $(y_i, \mathbf{X}_i)$, where $\mathbf{X}_i$ is a $3 \times n$ matrix of reals and $y_i$ takes a value in $\{1, 2, 3\}$. Essentially, $y_i$ represents a "selection" of the row vector ...
0
votes
3answers
93 views

Classification performance and the feature set selection

I am now working on a classification problem. The generated feature set can be separated into two group. I did a comparison study: use all of the features; use the features of group 1 only; and use ...
2
votes
1answer
105 views

penalized regression applications in epidemiology

I am seeking advice on penalized regression models for selecting covariates in epidemiological studies. A difficult tasks I face is feature selection while still attempting to account for confounding ...
2
votes
1answer
52 views

Should I exclude predictor variables if used to create a new one?

I have a dataset that includes race, gender, income, and family size. In addition, a variable for "sliding fee scale" tier is included, which is determined by income and family size. Should income and ...
0
votes
0answers
15 views

How to find entropy of vocabulary terms in multilabel document classification problem?

I have 5 million of document s with varying number of labels for each. I intent to find entropy value for selecting discriminative terms to degrade the size of vocab. However, having that multiple ...
0
votes
0answers
25 views

ARIMAX feature selection

I am implementing the ARIMAX model and need to implement feature selection. I have >100 features and a lot of data, so I need a method that isn't too computationally expensive. I tried a wrapper, ...
0
votes
1answer
145 views

Steps followed when Binary logistic regression when both dependent and independent variables are binary

I had set of binary variables. To apply logistic regression, I have checked association between dependent and independent variables and considered only those independent variables in the model which ...
4
votes
4answers
2k views

Term frequency/inverse document frequency (TF/IDF): weighting

I've got a dataset which represents 1000 documents and all the words that appear in it. So the rows represent the documents and the columns represent the words. So for example, the value in cell ...