Methods and principles of selecting a subset of attributes for use in further modelling

learn more… | top users | synonyms (2)

5
votes
2answers
775 views

Significance of categorical predictor in logistic regression

I am having trouble interpreting the z values for categorical variables in logistic regression. In the example below I have a categorical variable with 3 classes and according to the z value, CLASS2 ...
1
vote
1answer
285 views

Not all Features Selected by GLMNET Considered Signficant by GLM (Logistic Regression)

I wanted to create a predictive model of mortality after patients had undergone a surgical procedure. But I also wanted to avoid doing what most researchers do by first performing univariate analysis ...
1
vote
1answer
89 views

Dependent variables in regression

I have two variables in a regression problem, where I predict the end interest rate for a loan application. x1: risk band (A+,A,B,C) x2: initial rate (6%, 7%, 8%, 9%) Based on these two variables, ...
1
vote
0answers
29 views

How to use reservoir states for readout and training?

I’m trying to make a Liquid State Machine, I have a spiking neural network as the liquid, and a feedforward neural network that should learn to map the reservoir’s states to the output. I’ve read ...
0
votes
1answer
71 views

How to find important features in this problem

I was just thinking how ML techniques can be applied in the retail industry. Suppose we have data from a retailer who deals with apparel and cloth in this format and for each item there are ...
0
votes
1answer
198 views

What is a good Gini decrease cutoff for feature inclusion based upon random forests?

I am using random forests to try and determine variable importance as part of feature selection for a model I'm working on, and while I can get ranked variable importance by mean decrease in Gini from ...
0
votes
0answers
64 views

Topic Extraction using Scikit. Get documents for the Topics

I was trying to get a NNF model work for Topic Extraction. SCIKIT code http://scikit-learn.org/dev/auto_examples/applications/topics_extraction_with_nmf.html# works fine. But what i wanted was ...
8
votes
4answers
2k views

Test accuracy higher than training. How to interpret?

I've a dataset containing at most 150 examples (split into training & test), with many features (higher than 1000). I need to compare classifiers and feature selection methods which perform well ...
4
votes
0answers
65 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
3
votes
1answer
322 views

Can we use random forest for classification in combination with distance matrix between classes?

With a colleague, we are working on a dataset containing ~5000 continuous variables for 120 individuals belonging to 8 classes. We want to estimate the relative importance of each variable to explain ...
4
votes
1answer
317 views

Select best set of binary variables for clustering known sample labels

I have a set of samples, for which I know the "true groups". For this samples I have about 200 binary variables, I would like to know a method to select the subset of variables, that gives me a ...
3
votes
0answers
268 views

Variable selection / Dataset reduction for large datasets (in R)

I'm working on a behavoural scorecard modelling exercise, and many of the decisions taken to date have been based on the experience of a consulting credit analyst (whose experience software-wise is ...
0
votes
0answers
12 views

How to find the tendency of a feature in decision trees?

I've trained a decision tree binary classifier and I have the most informative features based on the sum of information gain weighted by the number of samples at the node (scikit-learn ...
1
vote
0answers
77 views

Using MatchIt to match groups in a retrospective analysis

I am interested in using the R package MatchIt to preprocess my data as to obtain matched groups based on a predefined treatment variable. However I am facing a few issues. The first issue is that ...
2
votes
1answer
207 views

Features selection using F-score for multiclass classification

I'm going to implement a feature selection algorithm, and I plan to use the F-score for because of its simplicity. The problem is that, the F-score is used for binary classification. How can it be ...
3
votes
4answers
4k views

Using LASSO from lars (or glmnet) package in R for variable selection

Sorry if this question comes across a little basic. I am looking to use LASSO variable selection for a multiple linear regression model in R. I have 15 predictors, one of which is categorical(will ...
1
vote
1answer
106 views

Evaluating features and similarity measures

I am currently developing a classificator, which is supposed to classify into a number of classes. For this purpose I am designing some features and similarity measures which I might use for a later ...
0
votes
1answer
372 views

AIC, BIC, DIC, model selection criteria

I am trying to understand the difference between these parameters, and their application. Was hoping to get some correction/clarification to my statements. I have a training set and cross-validation ...
0
votes
0answers
52 views

Comparison of feature selection algorithms

I have to compare some feature selection algorithms. Please suggest some study materials from where I can get the idea about the comparison method.
0
votes
0answers
33 views

Choosing prior distribution in LDA

how do you set prior distribution of K in LDA and can it be used for feature selection to improved selection accuracy of document. Abbey
1
vote
2answers
106 views

BMI at baseline & followup with exposure at baseline; model change or BMI at FUP? Control for BMI baseline?

For a prospective occupational cohort where everyone is exposed to one or more chemical agents, examining BMI at follow-up compared to a specific chemical exposure at baseline, is it necessary to ...
3
votes
0answers
132 views

How does LASSO select among collinear predictors?

I'm looking for an intuitive answer why a GLM LASSO model selects a specific predictor out of a group of highly correlated ones, and why it does so differently then the best subset feature selection. ...
0
votes
0answers
58 views

How many attributes to select for classification

I am aware that this question is vague, but after reading multiple standard books of the community, I am still wondering about the following applied problem. Lets say I have a dichotomy problem and ...
3
votes
3answers
3k views

Feature Selection Packages in R

I am very new to R. I am learning machine learning right now. Very sorry, if this question appears to be very basic. I am trying to find a good feature selection package in R. I went through Boruta ...
0
votes
1answer
170 views

Choosing one variable from each of 3 buckets of variables

I have a regression model that looks like the following glm.nb(formula = y ~ Gender + Age + x1 + x2 + x3, data = df) In my problem, there are 20 possible choices ...
0
votes
1answer
218 views

feature selection vs feature extraction

As per my understanding in dimensionality reduction, Feature selection chooses a subset from a list of available variables and, Feature extraction transforms available variables into lower dimension. ...
1
vote
1answer
38 views

Finding similar users

I am working on a problem in the online advertising space. I am trying to identify consumers similar to the set of consumers who have bought a product in the past (have 'converted'). If I can identify ...
1
vote
0answers
21 views

Is there any meta-approach for variable selection based of measures of similarity between each two variables?

Is there any meta-approach ( or mayby I should say universal approach which works with different measures ) for variable selection which is based on similarity matrix which every entry ...
0
votes
1answer
119 views

Relative importance weight with cforest

I am new in using RF. I want to use it to compute the relative importance of the features. I found the weight is very small ("party" package, cforest). Is there anyway to get these weights in a range ...
0
votes
0answers
57 views

similarity of feature vector

What is a better measure of feature vector similarity, Euclidean distance or dot product/cosine similarity? I've read about cosine similarity being used with document vectors, but I've also seen ...
1
vote
0answers
29 views

Performance worse with new observations

I come from the computer science area but am new to machine learning / stats, so this question may be fundamental and easy. I have time-series data (biological data), and, without getting into the ...
1
vote
2answers
404 views

Alternatives to glmnet for feature selection on data with lots of NAs

I have a surgical database in which there are approximately 100,000 observations and 200 features. Each observation corresponds to a separate patient while the features correspond to either ...
3
votes
1answer
179 views

Is it possible to compare two feature selections algorithms by cross-validations?

Assume I have two feature selection algorithms, A and B, which are developed based on SVM. I applied these two algorithms on the same dataset, a Liver Cancer dataset (400 features & 150 samples), ...
0
votes
0answers
131 views

Issues with sequential feature selection

I am trying to do some feature selection in gene expression data with 22215 features. I followed the tutorial here. I initially applied filter method(ttest) to select the features having the best p ...
2
votes
2answers
724 views

Mutual Information based feature selection

Suppose I have a feature matrix $F = [f_1^T,f_2^T,...,f_m^T]$ whereby $f_j^T \in \mathbb R^{n_t \times 1}$ is the $j$th column vector of $F$ ($n_t$ is the number of different events/trials and $m$ is ...
1
vote
2answers
701 views

Issues with feature selection in matlab

I am trying to use sequentialfs to do some feature selection in matlab. I have huge dimensional data of 22215 features. When I tried to use sequentialfs with svm as classifier so that it selects the ...
-1
votes
1answer
32 views

what are the dependent and independent factors and categorical for below project [closed]

my project is identification of factors influencing motor bikes for home to office trip. there are some attiributes like gender,age,marital status,education qualification,job type,working sector, no. ...
9
votes
3answers
298 views

What can cause PCA to worsen results of a classifier?

I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against ...
1
vote
0answers
81 views

True and false discovery rate in variable selection

I have a question about how I can calculate true and false positive rate in a simulation study? I have seen some articles and thesis by different definitions. One of them is the following one: ...
1
vote
2answers
117 views

Sparse hyperspace clustering

I have a dataset of M elements where every item is represented by a feature vector of length N where N is very large and only a small subset of N is bigger then zero for every item. So I have a sparse ...
3
votes
1answer
182 views

Variable importance randomForest negative values

I am asking myself if it is a good idea to remove those variables with a negative variable importance value ("%IncMSE") in a regression context. And if it gives me a better prediction? What do you ...
2
votes
1answer
162 views

How to perform step() when n < p in R?

I am trying to perform stepwise regression for variable selection in R. In matlab, the stepwisefit function is able to work in ...
2
votes
0answers
38 views

Why do correlations matter in simulating data to compare classifiers when p >> N?

In genomics and computational biology, expression data sets contain a much larger number of features (p) than the number of observations (N). I wanted to simulate data where p>>N to compare the ...
3
votes
1answer
486 views

Using Adaboost for feature selection?

Is it okay to use Adaboost to do feature selection (selecting a subset of dimensions $S$ from a high-dimensional feature vector $V$)? I divided the samples into four non-overlapping sets: $A$ ...
2
votes
1answer
70 views

A buggy but effective feature?

Through error analysis I found that a quite effective feature actually has bugs in its implementation. Correcting the bugs actually decreased the classifier's performance. What do you do? Correct the ...
1
vote
0answers
41 views

How do I know if the differences in ICs among candidate models are significant?

I'm doing some exploratory modelling on a data set with 29 covariates and an additional 11 variables that are of interest to my research question. My strategy is to develop a model with a subset of ...
11
votes
2answers
444 views

Bayesian variable selection — does it really work?

I thought I might toy with some Bayesian variable selection, following a nice blog post and the linked papers therein. I wrote a program in rjags (where I am quite a rookie) and fetched price data ...
3
votes
1answer
256 views

LASSO vs forward selection

I have two questions: I use cross validation to select a LASSO model, does the step in which a particular variable enter, indicate its relative importance? Let's age enter in step 1 and sex enter in ...
7
votes
4answers
756 views

Features for time series classification

I consider the problem of (multiclass) classification based on time series of variable length $T$, that is, to find a function $$f(X_T) = y \in [1..K]\\ \text{for } X_T = (x_1, \dots, x_T)\\ ...
0
votes
1answer
244 views

Feature selection with Caret for data with more than one target

I am trying to do some feature selection, having around 3500 variables for about 200 samples. To each sample is associated two numerical values (the expected outcome). I can't manage to make the caret ...